Comment by zahlman
2 days ago
> I'm not sure what you want me to do with that information
For example, you could cite specific things that you believe to be "AI tells" or "admissions".
2 days ago
> I'm not sure what you want me to do with that information
For example, you could cite specific things that you believe to be "AI tells" or "admissions".
It's a short article; you could read it. One example to get you started is the very first sentence:
> Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete.
The article is about how 'next-token predictor' is the wrong mental model; it opens with the admission that it is not the wrong mental model.
I did read it. People are allowed to disagree with your conclusions. Comment guidelines ask us all not to make such accusations.
To say that a statement is incomplete, but not strictly speaking wrong, is perfectly compatible with describing it informally as "wrong" in the sense used in the title (i.e.: "not the most appropriate possibility").
To informally describe something as wrong in an article focused on how it's wrong to informally describe something is incoherent.
There's a certain irony in pointing me towards the guidelines on the grounds that I have limited patience with your comments that violate them in various ways. I'm not sure that this is a productive discussion.
Not the person you’re replying to, but I read the whole article as an admission that it’s still a next-token predictor. More specifically: what does RLVR fundamentally change that somehow makes the whole process no longer a next-token predictor? The article makes no attempt to explain this. Additionally, I find its framing of the term “next-token predictor” as meaning “predicting the next token only based on raw training data” in common usage to be a bit dishonest.
To summarize: yes, RLVR and other synthetic training methods exist! It’s still a next-token predictor, and it does not “learn” or “think” or “reason” in the human sense, like so many people seem to believe.