Comment by daxfohl
24 days ago
I guess there needs to be a definition of "agent". To my intuition, the "agent" approach means multiple independent AI automata working in parallel and communicating via some async channels, each managing only its own context, each "always on", always doing something. The orchestrator is its own automaton and assigns agents to tasks, communicating through the same channels, mimicking the behavior and workflow of an engineering team composed of multiple independent people.
I see this as being different from a single process loop that directly manages the contexts, models, system prompts, etc. I get that it's not that different; kind of like FP vs OOP you can do the same thing in either. But I think the end result is simpler if we just think about it as a single loop that manages contexts directly to complete a project, rather than building an async communication and coordination system.
The bigger change is just to manage multiple contexts at all. I think how that is implemented will be determined through experimentation. I don't think the problems get much harder when you have multiple API requests in flight at once vs. doing them serially as you suggest. And for today's models, the speed increase would be nice, so it seems like it would be worthwhile.