Comment by Planktonne
2 days ago
I think you're trying to limit the meaning of both 'next' and 'prediction' in ways that don't reflect usage and that--if adopted--would severely limit our ability to discuss and evolve what LLMs are actually doing.
There's nothing inherent in either word that forces such a limit; predicting based on what will lead to success as measured by [reward function] is still a prediction.
It's not a prediction of the next move though, and that is the point. It's a prediction of what will happen if you make that move.
So, it's not a next move predictor. It's a game result predictor.
Eh, no, that's not right. I might need to brush up on my Sutton & Barto but the RL task is traditionally defined as, informally, "given a current state observation predict the next action, state and reward". A policy is always predicting the next timestep's reward. Otherwise, how would it know what to do next?
Brush up :)
The policy is optimized to maximize the the total reward, defined as the sum of the reward at each step, discounted by some factor.
2 replies →