Comment by cowanon77
2 days ago
It seems to be true this time though; I have observed it myself and heard it from several experienced developers I personally know and respect. It feels like some threshold was crossed with Opus 4.5 and Gpt 5.3, where the models are now able to reliably solve certain classes of problems that were previously unreliable.
Time will tell of course, and it’s early, but inflection points do exist with progress.
Thing is. You can find an extremely similar paragraph written about Claude 4.x or some equivalent gpt. And simultaneously, many people expressing their frustration and the shortcomings of <insert any model>
“But it’s different this time” - several people, several times over the last couple of years.
This is not at all a dig at you, I’m very sorry if it reads that way. My point is these things only get truly better in anecdotes. The ways in which they fail is yet to change. Just yesterday I had gpt 5.3 generate completely awful code for the Cinema 4D Python API. Also an anecdote. But for all of the people saying they are truly intelligent and truly reason, they still make obvious mistakes, write around problems, fail entirely at architectural decisions, fail at random, generate FAR too much code.
And no amount of harnesses, methodologies, loops make much of a difference. If you listen to people on the internet they say it’s all working. You listen to people on the job and they mostly say it’s creating tech debt and a review bottleneck. Also burnout, so much burnout.
I think LLMs are mediocre. I think it’s fine they’re mediocre. You can work with low expectations. But the hype cycles are so tiresome.
I don't buy into the huge LLM hype but I certainly think late 2025 was the inflection point. Up until then, I thought LLMs universally sucked at code. GPT-3.5, GPT-4, o1, o3, Claude 3.5, Sonnet 4, the whole bunch. Each iteration got marginally better, hallucinated APIs less and so on, but my overall evaluation of them all was that they wrote crap code and were unsuitable for anything other than one-off scripts.
Then the incremental improvements did, in my experience, cross some kind of threshold in late 2025 where the things became useful. It is of course anecdotal and personal judgment. But I asked LLMs to implement a small feature in my codebase (my usual test) and finally it produced code I was happy with. They've also been able to locate and diagnose a problem based on logs. In my view it's now a markedly different level of capability than we had a year ago, though I would call the previous two years equally useless.
You can find an extremely similar paragraph written about Claude 4.x or some equivalent gpt
Yes, as a product gradually improves there will always be many people for whom version X didn't work well for them and version X+1 does. It turns out that Opus 4.5 and GPT 5.1 were larger than average improvements that cross that threshold for a significant number of people.
My point is these things only get truly better in anecdotes. The ways in which they fail is yet to change.
If your claim is that there's no substantive difference between Sonnet 3.5 and Fable, then we live in very different worlds.
LLM capabilities are spiky. They're amazing at some things, and poor at others. Over time, the set of things they're amazing at has grown, while the set of things they're poor at has shrunk. If you think LLMs are just "mediocre" without any nuance, that's a sign you haven't spent the time to evaluate them in order to make an informed opinion.
GPT-5.3 is a last-generation model (which is wild to say when it came out in February this year, but it's true).
How does GPT-5.6 Sol or Claude Fable 5 or Claude Opus 5 do on that Cinema 4D code?
I believe it is a widely accepted opinion that agentic coding took off with Opus 4.5 late in 2025.
Why would you attempt to use GPT 5.3 to generate code today and form an opinion on that basis?
I do not think it is even still available in Codex, I believe it only has the smaller, distilled GPT 5.3 Codex Spark.
> You can find an extremely similar paragraph written about Claude 4.x or some equivalent gpt
Even 3.7. I remember when it came out and people were claiming that that was now the model that was going to replace engineers. Cue Fable years later and people still claim that this one is the one.
If it's so evident, why can't someone prove it with something more than "it seems better and everyone agrees"?
Have you ever seen evidence of a productivity improvement that you have found convincing for any software tool, independent of LLMs?
What shape did that evidence take?
Not shape, but even with napkin math, you can work out the conclusion. Take email vs snail mail. Online ordering vs relying on catalog. The spreadsheet, the word processor, the various CAD software.
The key thing is that it's easy to contrast the old way vs the new way and the evidence become obvious.
The thing with LLM tooling is that they're not reliable. I can do fine with risks, but only when there's a way to manage it so that if the disaster happens, it's practically a black swan event.
Typing more code or solving one task has never been the core problem. The core problem has always been to encode a whole system into the computer AND then provide a control interface for it. It requires both an understanding of the system you want to encode (especially how it behaves over time) and empathy to know what would be the best control interface for the users.
That understanding does not rely on the amount of code, and the best control is found through communication.
If we take the following project that you did:
https://simonwillison.net/2025/Jul/17/vibe-scraping/
An understanding of the system could be the following: A conference schedule consisting of events (time, place, speaker, description,...) stored or presented in some format. The interface would be: A web app with a mobile first UI that presents the information in an accessible manner (highlighting, filtering, exports,...).
A relatively quick (I haven't tested it), would have been to open the web inspector and extract the data using the dom API (requires knowledge of the dom api and a desktop browser), put the data into some json or a tsv file, then write a php script or a python script and then serve that. The interface could have been built with the standard elements of some css framework (bulma?).
Not saying the above is better. But the thing is that is doable from even a raspberry pi. And more it's repeatable and extensible. And the individual piece of knowledge are reusable in different situation.
You think the world is lacking in LLM benchmarks?
A benchmark doesn't prove "although we said it before, this time it's true". People pointed to the benchmarks then as well.
4 replies →
Con: "Where's the evidence that this actually is improving productivity?"
Pro: "The evidence hasn't shown up in statistics yet; it's too new!"
Con: "And won't this destroy maintainability?"
Pro: "Show me the maintainability disasters caused by AI."
Con: "I can't yet; it's too new!"
Both sides are playing the "it's too new" card when asked for actual evidence to prove their claims. In fairness, it actually is too new for there to be much statistically-valid data, especially if the inflection point was November 2025. So both sides are trumpeting their position, neither with actual trustworthy data.
Everybody has their anecdote. Nobody has data yet.
I wonder how much of it is real and how much of it is people just being worn down by the hype to the point they can't fight it anymore
Very smart people aren't immune to being worn down over time
I really don't think that's how it works. Smart, experienced developers who thought coding agents were junk for most of 2025 and think they're useful now in 2026 are not saying that because they got "worn down over time".
It tends to be when the training data wanders into their area of expertise temporarily and they go “OMG, they hype is real. I was so wrong” and then a few releases later they’re on the train and furious that the skills in their domain space have not just stopped improving, but regressed. Cue someone else in a different part of the world starting the same cycle.
Meanwhile the guy who leaned in a year ago and gave up reading the output is beginning to see work grind to a halt and throwing more agents at it is increasingly not working.
You can see these tropes all over social media near constantly.
6 replies →
As a smart, experienced developer who's getting worn down over time, I disagree.
I didn’t start using Claude Code until late 2025. Prior to that I would use ChatGPT to give me snippets of code but I was still doing most of the actual code writing. Coworkers told me in late 2025 about how they hadn’t written a line of code in “months” and just use Claude Code/agentic “whatever” so I tried out Claude Code and was pleasantly surprised. It is passable to have entire apps written by LLMs (I’ve made several that I otherwise never would have had the time to create by hand), but I wouldn’t say maintainable or easily extendable. It’s hard to be specific, but there’s something about LLM code that doesn’t look “natural”, and I’m not talking about the excessive use of comments in code. The code itself is unnatural. Functional, but unnatural. I wouldn’t want to suddenly lose LLMs and have to read through and understand and continue enhancing a codebase created by an LLM.
3 replies →
> Smart, experienced developers who thought coding agents were junk for most of 2025 and think they're useful now in 2026
The ones you make up in your brain don't count. I'm yet to see anyone serious reverse their claims.
11 replies →
You're being one of the voices trying to do the wearing down right now in this very thread imo
Of course it doesn't seem that way to you. Preachers view themselves as spreading the good word, they don't see how annoying it is being preached at
I was getting useful coding work done with GPT 3.5. I think devs saying "the models are finally good enough" this year are just trying to save face from their own previous irrational denials.
1 reply →