Comment by DoctorOetker

"You would never hand a newbie a 124 page policy document and expect them to correctly apply it on the first task, or to do it reliably in the first month"

So the behavior policy should reside in the weights, not in some ever expanding KV cache context, like sticking it in the wrong orifice if the goal is to be reproductive.

Instead of putting the policy document in some tightly crammed RAM, shove it in the weights that already exist, online / post training.

It does make one wonder if there is some way to compute the change to the weights from the computed context (so that we can free it up) without essentially continued "pre" training RMAD after each communication round.