Comment by YeGoblynQueenne

Sorry, I'm not convinced. "make_more_likely" is always maximising the probability of the next token given a sequence of preceding tokens. That's what we mean when we say "next token prediction", that the model returns the token that's most likely to follow the current sequence of tokens. The mechanism used to do that doesn't matter, it's still predicting the next token whether that's because it maximises a reward or because it follows a gradient or whatever else one might think.

Btw it's "make_most_likely" not "make_more_likely". It's just that "most" gets "mosted more" with every pass through the training data.

Edit: the article author's argument is that RLVR is different because it's learning to predict the next token by generating its own token sequences. That makes no difference: what is learned is still the conditional distribution P(token_k|tokens(1,...,k-1). That's the prediction task. Doesn't make a difference where you learned it from or how.