Comment by jdw64

Every time I use opus these days I go shut up... you are not fable.. Hard to imagine how just three days with it changed how I saw LLM use.

GPT-5.5 has been really hard to beat imho. I've spent $$$ on Opus, Deepseek v4 Pro and recently started to dogfood GLM-5.2 (which is not bad) but I cannot really trust any of them (almost blind) like I can trust GPT-5.5. It gives me tremendous confidence. I cannot say the same for any of the others I mentioned.

Yeah, Opus/GPT need multiple rounds of reviews from each other to get to clean auto review. Fable was like, it is done and indeed… crickets in bot comments. ‘No issues’ galore.

>> I am on the opposite camp. Open models are starting to perform better. GPT 5.5 keeps on messing things up.

I'm working in a 600k+ LoC codebase that has complex domain-specific logic and lots of moving parts. I find that Codex 5.5 is pretty good at surgical fixes, but does not go out of its way to explore and figure out what those surgical fixes might break. So I only use it to work on parts of the system that are pretty isolated from everything else so that risk of regression is small.

Ditto on GLM 5.2 + DeepSeek V4 Flash combo.

For most important work (complex, cross-domain inquiries etc.), I still rely on Codex GPT 5.5 though.

How are you running glm and deepseek? Local or hosted? If the latter, where do you run it?

how much does your setup cost you? just curious

Reading this thread makes me feel like I'm taking crazy pills. The folks on this train in my team do not produce anything significant that we can rely on or use. A lot of hollow prototypes that join the prototype graveyard and code that needs extra scrutiny on critical areas ultimately leading to taking longer.

It's a shame, they were smart and productive engineers. Now? I guess everyone is just all-in on the slot machine.

They claim extreme performance on ExploitBench, which Mythos was touted as being incredible at. https://x.com/OpenAI/status/2070555278576439306

Tracking model performance on Artificial Analysis makes me think these models are constantly optimized/tuned in some way or another. GPT 5.5 was scoring in the mid 60's when it was first released, now it's almost 10 points higher.

Maybe I'll know once I try it? Honestly, for small functions or methods, I don't think there's a huge difference between models. But the larger the code gets, the more noticeable the difference seems to be.

Personally, I think this kind of coding experience varies from person to person

sadly with all the labs benchmaxxing I feel like you just have to try the model for a while to really evaluate how good it is, especially for each individual use case

>zero coding benchmarks

"What gets measured gets managed"

> ... why was GPT's solution superior and faster?

Not saying that's the case with OP, but I've found folks sometimes just rationalize it so [0] as they're paying top dollar for it (especially, when compared to may be less capable but affordable models).

[0] https://en.wikipedia.org/wiki/Choice-supportive_bias

Not trying to be mean but it's likely the case that OP is not evaluating this properly, either due to a lack of skill or a lack of objectivity

For me in Game dev, codex has a habit of checking every argument for null and then silently early exiting the methods when true. I have explicit instructions for it not to do this - but it still does. I haven't done any c# outside game dev but I have no idea why people would want their programs to silently fail.

I think you could be right. I do use excessive error-handling code and verbose comments — that's true.

But most of my time is spent on delivery, and the biggest problem with delivery is that if a bug occurs during runtime, the client curses me out. So to me, GPT code feels meticulous.

Open source contributors might be different. Most of them write code after long periods of deliberation. They take their brightest ideas and put them into open source. Those pieces of code are probably the best answers those programmers can give.

But for someone like me, who works primarily on delivery, we mostly plug in proven patterns and focus on getting things done. 'It works' and 'it's beautiful' are different terms, after all. In that sense, I highly value the meticulousness of GPT code — the very thing you called verbose. Because even if it's inefficient, at least it runs, and it catches and wraps around far more of the parts where things break.

Given a month, I could probably write code at GPT's level, at least to some degree. The problem is the difference between one hour and one month. At its core, AI code is still based on training data.

When I searched for papers on using LLMs, I found that typically, you can have an LLM generate code and then ask it to find GitHub projects similar to that code. Then you can learn by looking at the pull requests and seeing how they structure things In the old days, if I wanted to understand why memory offsets, padding techniques, or data layout structures were written a certain way, I had to stare at a senior programmer's code all day or wait for them to reply. But LLMs, while they do flatter me, explain things at a level I can actually understand. And LLMs don't get annoyed.

There's a lot of tacit knowledge in programming.

-Why do you cut API boundaries this way? -Why do you change the order of struct fields? -Why do you deliberately insert padding?

Most of it depends on the background and context. Sometimes you add it, sometimes you don't. To understand this tacit knowledge, you need access to senior developers. But their attitude often depends on how promising the student is and what background they come from. On top of that, you don't have to rely on the respondent's mood, authority, or availability.

Programming is fundamentally a field that requires seniors. In my case, I had no such seniors at all. I learned to code by buying codebases from failed companies and studying them. My first job didn't hire me as an employee—they hired me as the CEO of a subcontracting company (because that was structurally more advantageous for the contract). So I wasn't given the patience to learn programming fundamentals gradually. I had to pay penalties if I failed. Most of the projects I worked on were the kind where failure meant bankruptcy for me. Naturally, there was no one to teach me.

Most of my knowledge comes from reverse-engineering the code I purchased.

People say LLM code contains falsehoods, but commercially sold code has always had falsehoods too. Honestly, if we're just talking ratios, LLM code has fewer falsehoods.

In that sense, I still think it's a matter of context. If LLM code is false, was human code ever really true? LLMs do lie. They generate plenty of incorrect code. But humans do the same thing. If a problem comes up, you just look it up then and there. For me, LLMs and humans aren't all that different.

You're not wrong. But programming isn't something only talented people do.

Another strong possibility is that you might be working on something that’s not very prevanlent in the training set.

Even the choice of programming language matters, e.g. Java or Javascript vs some niche one.

No offense but have you considered the strong possibility that you're just holding it wrong? You're entitled to your opinion, but OP is hardly the first person to say something like this and is surrounded by tons of folks saying the exact same thing. Just because it sounds weird to you, doesn't mean it's not true.

"no offense..."

... then says offensive thing.