Comment by rixed
1 day ago
I believe both of you are telling the truth. Certainly, you are not thinking about the same kind of tasks. For the gap between hairy realtime concurrent system programming, where a bad design decision can be fatal, and the simplest over engineered CRUD where nothing really matters can accommodate and explain the gap in shared understanding of LLM abilities.
> hairy realtime concurrent system programming
On the other hand, I’ve heard a direct quote from the longtime (25+ YoE) colleague of someone I know, that in their mostly embedded/some realtime work, the LLM code quality (specifically, this was about Opus 5.5 IIRC) has now surpassed his own. And that person is an excellent programmer, and the model is faster and cheaper!
So it’s definitely not just standard CRUD that is on the way to – or already – “solved”. As with all things, the truth is probably somewhere in between.
In my experience, it's more about "local" vs "large-scale" thinking. An LLM will be better at finding a bug that's "visible on screen" (or on the context window), whereas some design decisions that are hard to even put in words can be super important in practice (like, in C, how you manage the lifetimes of various objects, how you organise your threads and locks...) That's not purely "word manipulation", and there are not many good explanations of that on the internet, so it's understandably harder to learn for an LLM. A bit like chess vs go. They will get there quickly I'm sure. After all, AI beats us at go, too.
Our work is in embedded systems integrated with SCADA controls in a critical environment where data integrity relies on precise time synchronisation across sensor networks and databases. It’s an environment where we count processor cycles and joules , with very little room for error.
The difference is testing. Testing at every level. If crap code can’t move down the pipeline, you don’t get crap code. If a company is used to YOLOing code out the door with human developers and they try the same thing with LLMs they are going to have a bad day.
100% this, but also - linting, fuzzing, and test harnesses. Agents can help build out huge deterministic simulators and fuzzers that were simply too expensive / not feasible a year ago to build for most software - and then happily bind themselves to the results moving forward.
Agents just won't build it out by default - you have to ask for it.