Comment by jghn
7 hours ago
My largest issue is that when I'm looking for this I'm already dangerously close to autocompaction. And what I really want is a prompt which manages to preserve the most important parts of the chat log. And my opinion of important will not be the same as Claude, so we'll need to iterate on what that handoff really is.
I think what we really need is project/thread-scale continual learning. The problem is that the important parts of the conversation to you are the novel bits you just did, rather than all the context building the agent did to get to the point where it could do the novel bits (and even then, without really understanding the bigger picture).
If you snapshotted at 90% max context you could pretty reliably start iteratively trim that down, I think? I personally try to save the logs so agents can slice and dice them with sed/awk/jq/whatever when they need to look stuff up, because I’d rather pay the penalty on read (when it’s motivated by something) than in write(where you don’t really know what if anything will be needed), and they can figure out what they need on their own.
What I’d rather have is some way to bake history into the actual model weights (the same way it can recite certain literature or historical/factual stuff without context), with like multi-lora / “experts” that get trained out of band. But this is contrary to the “one fat model” approach to scaling and doesn’t work with closed labs’ business/IP models