Comment by roncesvalles
2 days ago
Usually, the longer the AI works on something the crappier its output because that means the context is getting filled up.
If my AI prompt takes more than 5 minutes to execute, I'm almost certain to stop it and break up the task or be more specific about what files it should look at or what subtask it should tackle first. The people who let it run for hours are absolutely not getting anything worth keeping in the end.
>Usually, the longer the AI works on something the crappier its output because that means the context is getting filled up.
This was definitely truer with older models but isn't necessarily the case now.
They frequently do other things apart from navel-gazing that take a lot of time but get good results, like spinning up subagents to solve some hairy task in a loop.
Stopping/distrusting long-running AIs is a habit I've had to unlearn myself.
I frequently get good results from a 30+ minute Fable session, when I've asked it to do something complex (e.g. run the QA tester in a loop and eliminate all crashes, one commit per crash fixed)
This is my experience as well. Lots of success with running one hour Fable sessions (one hour of API time). Not using a loop here, but there's often lots of documentation for Claude to go through first before building out the project.
The main failure point for me with long running Fable sessions now is just that it might hit a safety guardrail and downgrade to Opus midway.
Exactly.
I've found 6 minutes or so the sweet spot for upper bound with 5.6 Sol.
And it sounds like the OPs query above requires scanning throughout a large portion of the codebase, which will inherently consume a large number of input tokens. No locality to it.