Comment by Veedrac
2 days ago
The distinction is that it's not 'predicting the next token'. Instead it's _determining_ the next token based on a prediction of its reward signal.
2 days ago
The distinction is that it's not 'predicting the next token'. Instead it's _determining_ the next token based on a prediction of its reward signal.
> _determining_ the next token based on a prediction
Also known as predicting.
I think the most useful word in both cases is "extrapolating".
An LLM extrapolates from its context window to the immediate next token. This word applies whether you view what's happening as "reasoning", "prediction", or as a math function.
No, those aren't synonyms at all.
If I steer a car to avoid a predicted collision with a wall, this is not me 'predicting' the car. I am steering the car based on a prediction.
If you're assigning steering 70 of your 100 output points because it's what you think we should go with most of the time in this situation, I'm going to call that a prediction of how to steer.
2 replies →
Yes, but I think the same construction could also be used to characterize the first system; it determines the next move based on a prediction of its reward signal, where its reward signal is a measure of how likely it is that a grand master would make that move.
Like stanleykm, I found this analogy somewhat puzzling. On reflection, I think the author's point is this: the statistics of actual usage do not seem sufficient to produce a fluent LLM; it also takes reinforcement learning.
A classically pretrained LLM does not have a concept of having determined its previous tokens — it has only ever observed inputs that it had no causal influence over. This is why it's valid to say its actions are predictive and not determinative.