Comment by coffeebeqn
1 day ago
LLMs can create infinite worlds in the error message it’s receiving. It probably needs some outside signal to stop and re-assess. I don’t think LLMs have any ability to reason if they’re lost in their own world on their own. They’ll just keep creating new less and less coherent context for themselves
If you correct an LLM based agent coder, you are always right. Often, if you give it advice, it pretends like it understands you, then goes on to do something different from what it said it was going to do. Likewise, it will outright lie to you telling you it did things it didn't do. (In my experience)
So when people say these things are like junior developers, they really mean that they’re like the worst _stereotype_ of junior developers, then?
For sure - but if I'm paying for a tool like Devin then I'd expect the infrastructure around it to do things like stop it if it looks like that has happened.
What you often see with agentic systems is that there's an agent whose role is to "orchestrate", and that's the kind of thing the orchestrator would do: every 10 minutes or so, check the output and elapsed time and decide if the "developer" agent needs a reality check.
How would it decide if it needs a reality check? Would the thing checking have the same limitations?
Decision trees and random forests (funnily enough, this is not sarcasm).
You can maybe have a supervisor AI agent trigger a retry / new approach
They need impatience!