Comment by grim_io
6 hours ago
Maybe I'm misunderstanding, but this seems to oversimplify the problem.
Most frontier Models won't show you their true reasoning, but only a summary.
There is also no reason you can automatically assume the quality and price of the outcome based on the first few reasoning trace summaries.
If you look at the recent model releases, it becomes clear that the trend is that the largest models provide the most cost-effective solutions, because of the massive turn/tool call/reasoning reduction needed to find the solution, compared to cheaper and smaller models.
Our research seems to indicate you can get a decent signal based on the first few reasoning trace summaries. We analyzed hundreds of billions of tokens to figure this out. Any open-weights model also gives you reasoning, which is very helpful (because they are usually lower-quality candidates you would want to cut off early)
The largest models are very well known for having large reasoning traces. The thing that cracks down on overthinking is reasoning-effort training, which is largely a function of how the model is trained (certain RL objectives/teachers) rather than size of the model. The Kimi K3 technical report has some excellent work on this: https://arxiv.org/pdf/2607.