Comment by Phemist
13 hours ago
What is the intuition. Higher quality turns due to more reasoning results in significantly fewer turns taken?
13 hours ago
What is the intuition. Higher quality turns due to more reasoning results in significantly fewer turns taken?
Yep. In particular, ARC-AGI-3 is a series of games where if you fail, you keep trying again (until eventually hitting a timeout). So the sooner you succeed, the sooner you stop spending tokens retrying. If it was a benchmark where everyone got one attempt with no retries, you wouldn't see it bend backward.
Yes, in theory.