Comment by zerof1l
8 hours ago
This is totally a thing I noticed myself about 3 months ago. Medium thinking effort is ideal for most tasks. At high and above, models tend to generate more output in the form of comments or code for the same problem with no real benefit. Its a self-feeding loop: more output becomes more input, which then becomes more output. High is the highest I go. If I need more intelligence, it's better to use a more powerful model with less thinking effort or break the problem into phases. Much better result.
This version of Opus "max" apparently has even higher thinking output than Qwen "max", which is infamous for its thinking streams where it constantly second-guesses itself, then third-guesses, fourth-guesses and generally nth-guesses itself for arbitrarily large n. Of course, we aren't actually seeing Claude's raw thinking output: all we get is the after-the-fact prettified "summary". One wonders how much of that is a coincidence, or whether there's a reason behind that.
> Of course, we aren't actually seeing Claude's raw thinking output: all we get is the after-the-fact prettified "summary". One wonders how much of that is a coincidence, or whether there's a reason behind that.
Most of what I've heard is that raw reasoning traces are really good for distillation, although no idea how much the summarization actually hurts distillation.
There’s enough thinking leakage from the recent paper and just generally catching things on Reddit. Claude models overthink and self-doubt itself just as much as Qwen, but the summariser hides much of that.
Isn't this really bad news if you're even loosely concerned about so-called 'model welfare' and possible implications for alignment? The poor Claude is probably a lot more frustrated and paranoid than Marvin ever was, you just don't know about it because they actively force the model to pretend otherwise!
3 replies →
Personally I use everything in low reasoning. Maybe I'm wrong but I think that the higher reasoning settings are almost never worth it, it's marginal gains for a much higher budget.
I also switch to a better model for more complex tasks, also in low settings
Low is good if you’re working in a tight loop. But more risky for more agentic stuff you want to let cook for 30 minutes or more.