Comment by _345
14 hours ago
This makes so much sense as to why I've always felt that Opus 4.8 was leagues ahead of GPT 5.5. It's so good at taking underspecified requirements and filling in the gaps with sensible approaches for your project
14 hours ago
This makes so much sense as to why I've always felt that Opus 4.8 was leagues ahead of GPT 5.5. It's so good at taking underspecified requirements and filling in the gaps with sensible approaches for your project
Why supply underspecified requirements in the first place? Both models are good at challenging assumptions/edge cases and asking questions to clarify, but seemingly only when explicitly asked (i.e. something like a "brainstorm" skill).
I don't think either harnesses do enough to encourage the model to challenge all assumptions and ask questions, maybe because users might find it annoying. That step is basically a requirement IMO.
I've found all of the GPT-5 models to be very nit-picky, useful for code review and mathematics (important for my work), but seemingly gets in the way of "aesthetic" code, e.g. overly defensive code to cover all edge cases, even if unlikely.
There is seemingly also a tradeoff between flexibility vs instruction following. In my experience Opus will sometimes ignore instructions but can "fill in the blanks" more, vs GPT-5.5 follows instructions better but perhaps at the cost of rigidity.
> Why supply underspecified requirements in the first place?
Because you'd not want to forever loop outside your home when asked to "while you're out, grab some eggs" :)
Meaning why not leave home with your grocery list?
> Why supply underspecified requirements in the first place?
Because the entire reason we use LLMs is to supposedly improve productivity?
Refusing to sufficiently specify a task and hoping the model guesses correctly is not being productive. Again, these models still don't really ask questions when they should. You have to explicitly tell them to.
Specifying the problem is not extra work separate from solving it. If you skip that step, the ambiguity gets pushed into the model’s assumptions. Then you get a plausible looking answer to the wrong problem and have to waste time backing out of it.
LLMs are not magic machines that can read your mind.
5 replies →
> Why supply underspecified requirements in the first place?
Minimizes effort, is the obvious answer.
Poor trade off, the model is then designing a massive chunk of your solution instead of you. With a good spec, bits of typo’d pseudocode, and slightly more effort than a couple of sentences they can actually produce passable software.
I think the reason claude has so much mindshare is exactly because it’s more useful to non-developers who wouldn’t know how to describe what an api call executes to his grandmother.
For those who can, I can’t find much of a difference between them. Codex has the slight edge, but that’s all just “feels” to me.
1 reply →
The best benchmarks are the ones you create yourself.
Its not my experience Opus is leagues ahead or even superior, but in any case, since GPT 5.5 has Instant, Medium, High, Extra High and Pro...Should the comparison be with GPT on Pro, instead of Extra High as it seems to be the case in the table?
I didn’t know you could get the “Chat-GPT-5.5 Pro” (the one that’s been solving Erdos problems) inside codex-cli, or maybe I misunderstood?
And, in turn, Opus with ultracode?
Man I don't know if I'm living in a crazy bubble or something but GPT 5.5 is lightyears better than Opus 4.8 for me to the point where I'm honestly wondering how you're evaluating them or what kind of work you're doing.
There's specific tasks that Opus does better on like Frontend Dev and Design but for anything else 5.5 just laps it.
Yeah I’ve been consistently underwhelmed by anthropic models, but then I don’t use their harness so maybe that’s it
In my experience, for more mechanical refactoring work (like splitting a big source code file into multiple smaller ones), GPT 5.5 runs way faster than any of the Claude models. But for other tasks that require deeper reasoning, it's not that clear who is the winner.
It's just too funny to see people arguing about "no, it's my religion that's the right one!" on HackerNews.
You guys are all a lost cause.
How is attempting to benchmark llms like religion?
1 reply →
Better for vibe coders who always under specify. But at what point does it know you are under specifying but you have properly specified and it did it over your specification?
same observation here opus 4.8 (and i dont understand the people defending gpt 5.5 constantly) was significantly mature, it would even push back against anything off putting where as GPT 5.5 will happily agree and do what is asked but I would note that it takes several tries.
4.8 also requires more than one prompt but its output is significantly higher quality and offers more insight
Fable 5 is a different beast however.
> It's so good at taking underspecified requirements and filling in the gaps with sensible approaches for your project.
At a high level. It misses low level or other non-functional requirements differently so I wouldn't say Opus is just strictly better.
It's also possible that it's just a harness problem more than model.
I agree with you on the harness. I find that Claude can be good in any harness but GPT is only superior inside Codex.
Similarly, it explains to me why people found Claude so amazing, while I just thought "eh."
Tool expectations