Comment by zahlman
2 days ago
> generated an article that continually undermines its own main point.
I disagree that this accurately describes TFA.
2 days ago
> generated an article that continually undermines its own main point.
I disagree that this accurately describes TFA.
The entire second on chess engines is, from the view of the entire thesis of TFA, is incoherent. Let's assume, for sake of argument, that I agree with the section: that an idealized chess move predictor isn't a predictor — which is not a thing that exists, as the space of chess is enormous, but let's pretend! — that's not what LLMs are? Even if we just restrict ourselves to the space of written English prose, the space is quite literally infinite. So, hopefully obviously, no LLM is comparable to an idealized chess engine. Similarly, incoherently, we wave away the "make_more_likely", when, at least to me, the entire meat of that argument would be in the reward function, and we just gloss over that entirely.
(I would also agree with the parent commenter on that the writing smells like an LLM.)
The reward function seems indeed to be the protagonist there, still it stays in the shadows. One can only imagine that it is some kind of evaluator that scores the sequence based on grammar correctness, semantic consistency, etc.? To use the proposed chess analogy, maybe it could be a Stockfish engine that evaluates the submitted position that results from the move submitted by the LLM?
I'm not sure what you want me to do with that information; clearly I do think that my description is accurate.
The article is littered with both AI tells and admissions that 'next token prediction' is what is happening. Hence my description.
It was written by a human. There are AI edits but it’s very much a human composition. Perhaps a bit sloppy.
In my experience, people who do 'AI-assisted' writing tend to be very bad at noticing how much of their work AI has changed. I'm sure you put thought into it, but passing it through AI takes a lot of that out.
3 replies →
> I'm not sure what you want me to do with that information
For example, you could cite specific things that you believe to be "AI tells" or "admissions".
It's a short article; you could read it. One example to get you started is the very first sentence:
> Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete.
The article is about how 'next-token predictor' is the wrong mental model; it opens with the admission that it is not the wrong mental model.
2 replies →
Not the person you’re replying to, but I read the whole article as an admission that it’s still a next-token predictor. More specifically: what does RLVR fundamentally change that somehow makes the whole process no longer a next-token predictor? The article makes no attempt to explain this. Additionally, I find its framing of the term “next-token predictor” as meaning “predicting the next token only based on raw training data” in common usage to be a bit dishonest.
To summarize: yes, RLVR and other synthetic training methods exist! It’s still a next-token predictor, and it does not “learn” or “think” or “reason” in the human sense, like so many people seem to believe.