Comment by danielmarkbruce

2 days ago

The fight is about the predictor language in some cases. Because it's only a trivial difference to those who don't understand the details of how these things are made. In pre-training the model really is trained to predict the next token. What is being emitted by the model is, by structure, by training and by optimization, a prediction of the very next token.

What is emitted by a model during RLHF and RLVR is not, by structure, training or optimization, a prediction of the next token.

18 comments

danielmarkbruce

Reply
wat10000  2 days ago

Right. But that's never what the discussion is about. This is the first time I've seen someone say, "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." I have seen many, many people say "it's not just a next-token predictor because there's all this smart stuff going on inside the model." Even here, in the comments for an article that's all about the former, most of the objections to "next-token predictor" are of the latter form.

  • danielmarkbruce  2 days ago

    Yup, you are mostly right.

    I guess the people in my camp find the "it's just a next token predictor" stupid in that it's like saying "it's just a bunch of carbon and hydrogen", but it's also one of those things where people like to think they are clever because they think they are theoretically correct. But they aren't even that. So it's like double stupid. But the "next token predictor" part is at least technically correct (like, carbon and hydrogen right) for pretraining, so the debate can't really be won there.

    • Dylan16807  1 day ago

      It really is an important distinction, though. Being a next token predictor doesn't stop it from writing good sentences, but it does mean an LLM by itself can't play the number guessing game with you.

      5 replies →