Comment by hansvm
1 day ago
My colleagues write 10x less code for the same tasks (when they don't (ab)use AI) and don't reach past abstraction boundaries so that it's possible to actually reason about and review their work. When the problems get hard, they apply more care and attention to detail. Even if LLMs had the same bug rate per SLOC (IME it's higher), and even if those bugs were the same severity (IME they're worse), you'd still expect 10x bugs just because they can't be arsed to write a reasonable amount of code.
Almost all my code nowadays is AI-written, with me just contributing the concurrent data structures, a good spec, and some review and touch-up. I'm not _that_ much of a Luddite about this stuff. I'm not blind either though. Your "working together" comment does a lot of heavy lifting when arguing against a post saying that leaving the LLM to its own devices is a bad idea.
If you are reaching past abstraction boundaries and making spaghetti code you are holding it wrong.
We create higher quality, more testable code, with less complexity and less LOC per functional unit than we ever did with humans. By a long stretch.
Documentation driven development with constant naive review, delta review, documentation/code coherence review agents, test coverage review agents, development standards and architecture review, etc. We write code once, then review and revise each module from a variety of different paradigms until every line, every construct, every function, every interface is optimally constructed and documented. Agents learn to write code that passes with less flags by having it pushed back to them until it passes, and revising their learnings files.
By the time code makes it to a commit it’s been brought into compliance from a variety of perspectives and tested as completely as possible, along with mutants that test the tests. By the time it gets to a merge gate, most code has been through at least 50 reviews. Then we periodically sweep the entire codebase for cruft, vulnerabilities, corner cases, etc. If we find smells or clever sections, they are eradicated.
In total, the hot paths and critical infrastructure sees hundreds of reviews against thousands of rules and best practices.
Sure, it’s expensive, but still 1/4 the cost of regular development, and much, much fewer bugs.
Unfortunately, most people are out there just vibin, even at the big corps, and it’s a dumpster fire.
It’s easy to produce garbage software, but it’s not mandatory.
I believe both of you are telling the truth. Certainly, you are not thinking about the same kind of tasks. For the gap between hairy realtime concurrent system programming, where a bad design decision can be fatal, and the simplest over engineered CRUD where nothing really matters can accommodate and explain the gap in shared understanding of LLM abilities.
> hairy realtime concurrent system programming
On the other hand, I’ve heard a direct quote from the longtime (25+ YoE) colleague of someone I know, that in their mostly embedded/some realtime work, the LLM code quality (specifically, this was about Opus 5.5 IIRC) has now surpassed his own. And that person is an excellent programmer, and the model is faster and cheaper!
So it’s definitely not just standard CRUD that is on the way to – or already – “solved”. As with all things, the truth is probably somewhere in between.
1 reply →
Our work is in embedded systems integrated with SCADA controls in a critical environment where data integrity relies on precise time synchronisation across sensor networks and databases. It’s an environment where we count processor cycles and joules , with very little room for error.
The difference is testing. Testing at every level. If crap code can’t move down the pipeline, you don’t get crap code. If a company is used to YOLOing code out the door with human developers and they try the same thing with LLMs they are going to have a bad day.
1 reply →
>It’s easy to produce garbage software, but it’s not mandatory.
It is not mandatory, just like doom scrolling is not mandatory. But given human nature, both are inevitable...
How is that 1/4th the cost of the regular development? Did your pre-AI code ever hit 50 reviews? If not, it’s impossible that it was slower. If you are talking about agent reviews, it’s not expensive.
Which is it?
I'm not the guy you're responding to, but I have the same experience. If anything, 1/4 cost underestimates the savings.
The analogy is like pointing out that you can have a robot vacuum clean your floors every day. Before the robot, you'd do it maybe once a week, just no time - "too expensive" to even think about doing it daily. The robot makes it happen all the time for a tiny, tiny fraction of the cost.
6 replies →
You're kind of saying the same thing as the parent but in a smug, insulting and disagreeable way. Many acknowledge LLMs are very useful but without needing a jargon-spew that kind of "smells" like LLM output itself.
Sounds great! Let’s see the code.
AI's default state of affairs is to reach past abstraction boundaries and apply whatever cheats it needs to pass its tests quickly. I think your argument is that sufficiently smart harnesses can mitigate that defect. I disagree, but not as much as you might think.
You can (and should IMO) apply additional guardrails to generate better code. If I'm interrupted every 10min to review garbage then the AI literally doesn't make me more productive, and I'd be better off coding normally and using it as a glorified stackoverflow/searchengine. This isn't how I see other people using the tools (largely because of costs AFAICT) -- which is absolutely part of my complaint; people are holding it wrong, and I'm left to pick up the mess -- but I personally refuse to deign to answer an AI's question till my normal list of problems is addressed. Nevertheless, I still disagree with your conclusion, that automatically addressing a small enough list of issues to fit into an agent's context results in good code.
I'm not sure if you're interested in a challenge, but one hand-coded thing we did well at $WORK was writing the fastest protobuf parser for this schema (and, generally, any other messy, deeply nested schemas) [0] (assume many adcandidates, deals, and segments if you're generating synthetic data to test against). My ranked priorities are (a) I need to be able to easily prove it correct/safe for all inputs (literally all, not just the few hundred billion we received yesterday, since this is open to the public internet), (b) constrained by (a) I need it to have as much throughput as I can possibly manage for many of my workloads, and (c) a second implementation optimizing latency instead of throughput is used for the rest of our software. AFAICT, AI is currently incapable of even understanding what the bottlenecks and priorities are. When I manually describe those, its solutions are a dumpster fire. If I describe exactly what I want coded in excruciating detail then it does reasonably well (not by default, but with enough of the automated tests you've described it doesn't take too many iterations with a person for it to be shippable), but at that point I've already done all the hard work, and if it took 2 weeks of understanding and design and then 1 day of AI coding vs 2 days of human coding then I haven't really bought much.
A full solution seems like a tall order for some rando's internet challenge, but are you interested in seeing if your AI can even identify the relevant problems and propose plausible solutions to them? I'll make sure to revisit this comment thread and reply with critiques and follow-ups if you're interested.
[0] https://github.com/BeeswaxIO/beeswax-api/blob/master/beeswax...