Comment by coder-pm
13 hours ago
That kind of one shot capability is impressive but how does it work for my typical work style? The way I work is to build a huge roadmap with goals and hand it to my agent to execute (often over night). I don't care that much about the benchmarks, what I care about is how often Fable 5.1 is making a baffling decision and destroys my plan, not respecting stop conditions or goals. I would seek for behavioral reliability over long autonomous runs, not eval scores. Anyone have that kind of feedback and observations?
You can engineer loops that have it, but it depends on a case by case basis. Does your loop have strong validation? if it's all vibes nothing can stop it from diverging.