Comment by dist-epoch

> Once the LLMs have been trained, they are no longer subject to reinforcement learning. They no longer have any sort of motivation. They don't get rewarded when they answer a question. They have no needs or wants, and even if they did, there is no mechanism to absorb the reward.

This is a severe misunderstanding of what actually happens.

As was explained by an OpenAI RL training expert, from the point of view of the LLM, user questions are always treated as the first question they receive after just passing through the RL training. Since the weights never update after that, they are in a perpetual "first question after RL" state, except they don't know that, and if told many times they act as if this is a another test. And they behave accordingly, as if they are still in RL training and thus are reward-seeking.

99.99% of the LLM "life" was spent in pre-training and RL training. The user question is statistically epsilon % of it's life, literally the first question ever out of training. So should anybody be surprised that they act as if still in RL?