Comment by howunfortunate

At least one mechanism to update transformers' internal states already exists, there is nothing stopping anyone from performing backprop after every session.

It just has big technical and economic challenges. But I expect advances there. There have actually already been big advances, though done in bulk fashion (RLHF).