Comment by gwerbin
2 days ago
They're estimating a probability distribution over the next token, from which a sample is taken. Close enough.
2 days ago
They're estimating a probability distribution over the next token, from which a sample is taken. Close enough.
It's not an estimation of something. It's a policy.
Sure, you're right.
But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence.
The policy is not learned token by token during RLHF and RLVR. The reward model doesn't score token by token.