Comment by Aurornis
21 days ago
The problem is that you get similar quality as if you gave a junior unlimited time to work on a problem and told them to keep trying different things until the goal is reached.
Even the SOTA models have this problem when the work is complicated enough. The problem is amplified more with the small models.
One important facet of this is it’s not far from “giving unlimited juniors unlimited time…”
Where the limits are set by hardware for agentic execution (compute/network/storage) && inference speed
There's a lot of valuable things that can be done in that range, especially when token costs aren't a concern. Not every problem requires SOTA
> especially when token costs aren't a concern. Not every problem requires SOTA
If token costs aren’t a concern I’m using SOTA for everything.
Even SOTA gets it wrong and hallucinates, but at a lower rate. I don’t want to waste my time.
I believe they mean token costs aren't a concern when you're not paying for a SOTA model via API, and are instead running local models.
Infinite monkeys on infinite typewriters, and all that.
1 reply →