Comment by daxfohl
8 hours ago
There's no need to anthropomorphize though. One loop that maintains some state and various context trees gets you all that in a more controlled fashion, and you can do things like cache KV caches across sessions, roll back a session globally, use different models for different tasks, etc. Assuming a one-to-one-to-one relationship between loops and LLM and context sounds cooler--distributed independent agents--but ultimately that approach just limits what you can do and makes coordination a lot harder, for very little realizable gain.
The solutions you suggest are multiple agents. An agent is nothing more than a linear context and a system that calls tools in a loop while appending to that context. Whether you run them in a single thread where you fork the context and hotswap between the branches, or multiple threads where each thread keeps track of its own context, you are running multiple agents either way.
Fundamentally, forking your context, or rolling back your context, or whatever else you want to do to your context also has coordination costs. The models still have to decide when to take those actions unless you are doing it manually, in which case you haven't really solved the context problems, you've just given them to the human in the loop.