Comment by gwerbin
1 day ago
Sure, you're right.
But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence.
The policy is not learned token by token during RLHF and RLVR. The reward model doesn't score token by token.