Comment by gessha
4 days ago
I’ve been going back and forth between opus and local Qwen 27B and a good number of times opus makes absolutely stupid things compared to Qwen.
For a large number of tasks you definitely don’t need a frontier model but a good harness around it.
If you don’t can’t/want to experiment with local model, have Claude implement the exact same feature with haiku/sonnet/opus and see what the differences are. You might be surprised.
I've used extensively everything between Opus 4.6, GPT-5.5, Fable 5, Opus 5, GPT-5.6 Sol, GPT-6 Astra, 6.1 Sol and Opus 5.5.
To a lesser degree also Sonnet and Luna.
And when I say extensively I mean 500h+ on a side project in addition to my full-time job.
Even the difference between 5.6 Sol, which is miles ahead of local Qwen 3.8 27B, and Opus 5.5 is immediately obvious in the quality of the results.
Having been stuck on 5.6 Sol at my job while Opus 5.5 was available at home, I can say with full confidence that the difference is significant, even if I weren't looking at the output at all.
The way it understands my requests, forms a hypothesis, verifies the result and finishes the change is night and day different. This happens while 5.6 Sol is still flailing around somewhere in the monorepo struggling with even locating the relevant code.
I took a break from commercial plans and was using local qwen/glimmer/gemma and hosted glm/DS4.1-flash for things; opus 5.5 is night and day difference to return to. It makes some mistakes certainly, but its commits are often exactly the change I would have made with far less babysitting. It also shows that fable spark of common sense of “hey…you realize this other thing is broken too right?” without getting lost in the weeds immediately like other models seem do. Still an idiot sometimes though... “Oh no, I’ve discovered the exact same thing we have discussed and you have approved 5 times, and I’ve made notes about…I don’t know what to do, can you tell me to read my own note I’ve just reviewed and ignored?”
Exactly. And without setting a /goal for the harness to loop it, it also did essentially this today after a few hours of work:
It's also happy to write multiple paragraphs about different implementation options, where the ticket's acceptance criteria already rules out all but one.
The outputs are still often verbose and the language always is convoluted Claude-speak.
Even for typical day-to-day software engineering work, there is still much room for improvement.