Comment by jorvi
2 days ago
> zero shot
I really wish we would find a different term for this.
Doing something always takes at least one attempt, i.e. "one shotting". "Zero shotting" is an oxymoron, which makes it a term that only creates more confusion rather than succinctly conveying something.
"One shot" is simply about the action itself, but it says nothing about how much preparation was done beforehand. "Zero shot" additionally implies without training or preparation.
TCGs have a related "zero turn win" concept, where the opponent goes first and you win without getting a turn due to the set of cards you randomly drew and being able to activate them on the opponent's turn.
I think of a shot as an example, not a try: “One shot” is “One example”. Zero shot is “Zero examples”. I don’t love it, but I don’t hate it, got a better word for it?
We already have a term for it in people, "intuited". When you are asked to intuit something, it usually implies an unfamiliarity with the subject matter.
There is such entrenchment with terms though, it'll never get shifted to that.. and on top of that, it doesn't sound as interesting and dynamic as "zero shotting".
to be fair, it's also pretty long winded to say "pass @ 32 attempts to intuit" or "intuited after 6 examples"
I mean... how about "example"? I feel as if I were to give what you just said to someone a hundred years ago, with no context of AI training or even of the discussion, the very form of what your response leads to the answer "example" ;P.
The issue with "shot" is that it is a term and part of an idiom that has been used for a very long time and, critically, is relevant to the same problem space in a much more intuitive way: to count the number of shots shot, not shots seen.
We say Sure Shot.
It's a shot from position zero
No it isn't. The number of shots (examples) is zero.
You're both right.
My favorite AI term to ridicule is the recent "Test Time Compute" nonsense, which has nothing whatsoever to do with testing. It literally just means "inference time".
And if I hear someone say "banger", "cooking", "insane", or "crazy", one more time I'm going to sledge hammer my computer. Can't someone, under 40 please pick up a book and read. Yesterday Sam Altman tried to coin "Skillsmaxxing" in a tweet. I threw my coffee cup at my laptop.
Speaking of old-timers and "inference time" - there was a time when "inference" meant inferring parameters from data (i.e. training). And now it means "test-time". (or maybe the difference is if it's statistics community vs ML community).
e.g. Bishop's textbook says:
5.2.4 Inference and decision
We have broken the classification problem down into two separate stages, the inference stage in which we use training data to learn a model for p(Ck|x) and the subsequent decision stage in which we use these posterior probabilities to make op- timal class assignments.
I almost mentioned "inference" too, as an unfortunate word that stuck in a bad way, but it's tolerable since we can now just [falsely] claim that the AI is "inferring" what a prompt "means" in order to answer it.
And speaking of word definitions: "Old Timer" is anyone with a decade more experience than you.
It makes quite a lot of sense juxtaposed with "train time compute". The point being made is that a set budget can be split between paying for more training or more inference _at test time_ or rather _at the time of testing_ the model. The word "time" in "inference time" plays a slightly different role grammatically (noun, not part of an adverbial phrase), but comes out to mean the same thing.
Exactly right. The term "Test Time" had relevance in a certain context, and in a certain paper, but once people read the paper and saw the term they latched onto it, not realizing how totally non-descriptive and nonsensical it was when used outside that specific narrow context of genuinely "testing".
Get off my lawn is alive and well it seems
Speaking of worn out tropes, you just used the most common one of all. I'm sure it was a tough call for you to decide between that and a "boomer" quip.
Array indexing can start at 0 or 1.
For an array of zero shots, the indexing doesn’t matter.