Comment by SamPatt
1 day ago
It's not that we're assuming the AI has made no mistakes. It's that we're confident in the general capability of the models such that we're willing to accept it'll make some mistakes, but working together with it, we'll still achieve our end goals.
I presume you also know your own code has mistakes, too? You wouldn't refuse to even look at the code of a colleague who has ever made a coding mistake, would you?
My colleagues write 10x less code for the same tasks (when they don't (ab)use AI) and don't reach past abstraction boundaries so that it's possible to actually reason about and review their work. When the problems get hard, they apply more care and attention to detail. Even if LLMs had the same bug rate per SLOC (IME it's higher), and even if those bugs were the same severity (IME they're worse), you'd still expect 10x bugs just because they can't be arsed to write a reasonable amount of code.
Almost all my code nowadays is AI-written, with me just contributing the concurrent data structures, a good spec, and some review and touch-up. I'm not _that_ much of a Luddite about this stuff. I'm not blind either though. Your "working together" comment does a lot of heavy lifting when arguing against a post saying that leaving the LLM to its own devices is a bad idea.
If you are reaching past abstraction boundaries and making spaghetti code you are holding it wrong.
We create higher quality, more testable code, with less complexity and less LOC per functional unit than we ever did with humans. By a long stretch.
Documentation driven development with constant naive review, delta review, documentation/code coherence review agents, test coverage review agents, development standards and architecture review, etc. We write code once, then review and revise each module from a variety of different paradigms until every line, every construct, every function, every interface is optimally constructed and documented. Agents learn to write code that passes with less flags by having it pushed back to them until it passes, and revising their learnings files.
By the time code makes it to a commit it’s been brought into compliance from a variety of perspectives and tested as completely as possible, along with mutants that test the tests. By the time it gets to a merge gate, most code has been through at least 50 reviews. Then we periodically sweep the entire codebase for cruft, vulnerabilities, corner cases, etc. If we find smells or clever sections, they are eradicated.
In total, the hot paths and critical infrastructure sees hundreds of reviews against thousands of rules and best practices.
Sure, it’s expensive, but still 1/4 the cost of regular development, and much, much fewer bugs.
Unfortunately, most people are out there just vibin, even at the big corps, and it’s a dumpster fire.
It’s easy to produce garbage software, but it’s not mandatory.
I believe both of you are telling the truth. Certainly, you are not thinking about the same kind of tasks. For the gap between hairy realtime concurrent system programming, where a bad design decision can be fatal, and the simplest over engineered CRUD where nothing really matters can accommodate and explain the gap in shared understanding of LLM abilities.
4 replies →
>It’s easy to produce garbage software, but it’s not mandatory.
It is not mandatory, just like doom scrolling is not mandatory. But given human nature, both are inevitable...
How is that 1/4th the cost of the regular development? Did your pre-AI code ever hit 50 reviews? If not, it’s impossible that it was slower. If you are talking about agent reviews, it’s not expensive.
Which is it?
7 replies →
You're kind of saying the same thing as the parent but in a smug, insulting and disagreeable way. Many acknowledge LLMs are very useful but without needing a jargon-spew that kind of "smells" like LLM output itself.
Sounds great! Let’s see the code.
AI's default state of affairs is to reach past abstraction boundaries and apply whatever cheats it needs to pass its tests quickly. I think your argument is that sufficiently smart harnesses can mitigate that defect. I disagree, but not as much as you might think.
You can (and should IMO) apply additional guardrails to generate better code. If I'm interrupted every 10min to review garbage then the AI literally doesn't make me more productive, and I'd be better off coding normally and using it as a glorified stackoverflow/searchengine. This isn't how I see other people using the tools (largely because of costs AFAICT) -- which is absolutely part of my complaint; people are holding it wrong, and I'm left to pick up the mess -- but I personally refuse to deign to answer an AI's question till my normal list of problems is addressed. Nevertheless, I still disagree with your conclusion, that automatically addressing a small enough list of issues to fit into an agent's context results in good code.
I'm not sure if you're interested in a challenge, but one hand-coded thing we did well at $WORK was writing the fastest protobuf parser for this schema (and, generally, any other messy, deeply nested schemas) [0] (assume many adcandidates, deals, and segments if you're generating synthetic data to test against). My ranked priorities are (a) I need to be able to easily prove it correct/safe for all inputs (literally all, not just the few hundred billion we received yesterday, since this is open to the public internet), (b) constrained by (a) I need it to have as much throughput as I can possibly manage for many of my workloads, and (c) a second implementation optimizing latency instead of throughput is used for the rest of our software. AFAICT, AI is currently incapable of even understanding what the bottlenecks and priorities are. When I manually describe those, its solutions are a dumpster fire. If I describe exactly what I want coded in excruciating detail then it does reasonably well (not by default, but with enough of the automated tests you've described it doesn't take too many iterations with a person for it to be shippable), but at that point I've already done all the hard work, and if it took 2 weeks of understanding and design and then 1 day of AI coding vs 2 days of human coding then I haven't really bought much.
A full solution seems like a tall order for some rando's internet challenge, but are you interested in seeing if your AI can even identify the relevant problems and propose plausible solutions to them? I'll make sure to revisit this comment thread and reply with critiques and follow-ups if you're interested.
[0] https://github.com/BeeswaxIO/beeswax-api/blob/master/beeswax...
This is a different argument from Karpathy's prompt-and-forget one.
I personally have had good success with using agents as virtual pair programmers following some simple rules for how to make that process productive, but that is a very, very different approach. One that recognizes that both humans and agents are prone to errors and attempts to mitigate the problem rather than expecting it will magically work out through the power of AI.
I think of it in generational terms. Opus 3 required me to prompt and edit, iterating over functions almost one at a time. Opus 4 started to show agentic capacity but it largely bloated code and still needed frequent course corrections. If it churned for more than 15-20 minutes you knew something was off.
Opus 5 takes that and changes the game. A simple prompt goes through research, design, verification, review and self-corrects constantly. Agentic workflows can run for days without stopping, without losing the thread or devolving into slop.
I can’t even imagine what Opus 6 is going to bring.
The biggest thing holding AI back is people. People can only change so fast and yet AI doesn’t care. It just keeps accelerating.
I am sorry, but current benchmarks do not bear out that claim. Yes, frontier models can fairly reliably one-shot standard CRUD apps (and you don't even need frontier models for that), but when we're getting to harder problems, reliability nosedives when they're operating autonomously.
I am shocked how many bugs AI finds in the gray bearded wizards code at work. Just tons of stuff I didnt catch and I assumed wasnt there.
I’ve had this experience as well where it is finding all kinds of subtle bugs. The trouble is when it lulls you into checking its work a little less carefully and you let some of its bugs slip through as well. Definitely had a few of those happen to me at work.
The problem is also that enticement thing it does. Over representing the importance of some bugs or introducing more complexity to squash something that doesn’t matter.
If you aren’t 100% on it the entire time it’ll make 4 bugs for each solution and then write tests that hand hold the nonsense through. The juice isn’t always worth the squeeze and it messes with your ability to parse it all.
1 reply →
Being a gray beard myself. I am hardly surprised, we used to find many of these through users in production.
Yes, working on a customer's 20 years old software architecture in logistics it's fascinating in a horrifying way how many of those "it only affects 1-100 supply chains" bugs are in production.
It's less gray bearded wizards than the fact that they used cheaper crazy cackling goblins to write the software.
The challenge is to determine if a finding is an important bug or just a dead business process now never called in production. That is knowledge the AI does not have, and it requires careful, exhausting work of going through the problem.
i mean, you way is certainly one way..
but ai is perfectly capable of adding/enable matching monitoring so you can check what processes are used... or ai can
The amount of crazy stupid bugs in the 8-year-old codebase I work on is shocking. Every time I work on a new subsystem I end up fixing 20 incidental bugs; lots of N+1 query problems, completely wrong type annotations, all kinds of things.
I was under the impression that the point of strict types was that the code wouldnt compile/run with wrong type information.
3 replies →
> type annotations
Stopped reading right there
Are the bugs there though, or the LLM says there are?
LLMs are no trained to say "no". They are likely to find a ton of bugs in a simple "Hello, world" than generate "no, there are no bugs".
On top of that: if there are possible bugs in an isolated piece in code they find, are those bugs actually there?
They're there, but in my experience, lots of them are for edge cases that almost certainly wouldn't have been hit in prod. I think this is a major reason why LLM-generated code can be much longer for the same problem than human code. Humans won't think of every edge case, and occasionally that bites them in the butt, but they also won't cover all the edge cases that happen rarely enough that they will not matter.
I'm unsure what the right solution is. I currently tend to let LLMs resolve these bugs because it does fix genuine issues that would have affected real users. But this also means that LLM-generated code is often twice as long as what I would have written, and much of it is for defensive behaviors that will never matter in prod.
2 replies →
That's not really my point... my point is that it's an unsettled debate between speed and caution, and devs are not unanimous on it, unlike the claim of the original post that seemed to think everyone was reporting the same gains.
The divide with developers at the moment seems to be about people who believe in review & are being cautious (who are not seeing major time gains according to all reports I have seen), and those who trust it to be able to fix things forever and will no longer have to worry about understanding the code, so are happy to atrophy those skills.
Yes, my code will have mistakes, and I'll use LLMs for some tasks. I also review and have a mental model, which means I can easily fix them, no matter what happens in future with AI. Reviewing also means I'm seeing the limitations and common mistakes of LLMs.
A no-review approach will be far faster than this, obviously, but if scbench.ai is still true on the frontier models, it may come back to bite people when they have no idea how it works and the context becomes too big to maintain.
I think a big part of that divide is between the people who are inherently limited by other factors on the rate of change they can implement.
If I'm being snarky - the people who are happy to run without reviewing the code probably mostly are able to because there isn't significant value attached to it downstream. Not many stakeholders, not high value transactions, not complex downstream integration or impact. And yes, in their friction free lives, AI can massively accelerate things far beyond what it can do for the rest of us. But don't infer that applies everywhere else where those assumptions don't hold.
Would disagree on the terminology, but not the spirit.
I’m sure people are building things that have specific KPIs based on client preferences. As long as those preferences are being met, then their question becomes “why does it matter what the code quality is?”
Perhaps it’s simply liability and penalties? If getting something wrong means your client is going to sue you into oblivion, then perhaps that is where people are focused on making sure its right, as opposed to making sure its fast?
1 reply →
Asking AI to review my code is actually pretty useful... but makes the process slower.
Yup. Also code review is not just there to check your work. It's forced communication, in that at least someone else knows about the changes you're making.
It's also a critical part of your security model. "Nobody can deploy code by themselves" is shockingly effective at preventing rogue actions.
Speaking of communication... Almost no one mentions talking to non-devs. That's one of the reasons I can't just rely on any of these tools, I have to remember and understand everything well enough I know how they work and can explain things and help plan ahead when talking to product people in meetings.
1 reply →
[dead]
> You wouldn't refuse to even look at the code of a colleague who has ever made a coding mistake, would you?
Not sure that is the position you're arguing with. I would refuse to skip code review for any colleague, no matter how trustworthy they were. This is the behaviour we are expected to accept to get productivity gains, when the colleague is an LLM.
Last week, on my pet project that I am using a harness to almost entirely code (so I'm by no means a Luddite), it/Astra decided to dump ALL application state into a single row+column in SQL as JSON. That is a mistake that I would, and a completely new grad, would _never_ make. It is profoundly incompetent.
This week it did it a second time on an unrelated project.
"Some mistakes" is putting it unbelievably lightly. It is shitting out complete and utter garbage, and if you think it's doing a reasonable job (or a job that can be fixed later by a more powerful model), you're either not paying much attention, or your own ability has already heavily atrophied.
Karpathy put it well: "gap of understanding of LLM capability," but he's seemingly in one of the less fortunate groups himself.
Imagine if other engineering disciplines just started accepting that everything they built or worked with would be only 98% correct.
Over time, the errors don't "work themselves out together," they compound. Everything would stall in a mess of unreliability and chaos.
Civil engineers are routinely called to work with materials like dirt and concrete whose properties cannot be guaranteed to much beyond 98% precision.
2% tolerance is not the same as 2% failure rate
I don't quite get what's the argument you're making here. What's in contention is whether or not the 0.2% is in agreement.
Are WE the 0.2%?
Because if not, what's the point talking about the merits of AI's competence in of itself? Accomplishes nothing, we're he outsiders. And if we are, then either you're fishing for disagreement and the post you replied to is actually correct, or you get unanimous support to your statements, and establish... Nothing, the in-group could be delusional in a way that does not actually function as an argument against the whole "So where's the software? We're the AI led market disruption instead of the AI-led shovel makers?". The punchline was about the clients being unhappy.
This attitude is exactly why AI bros don't make successful software lol. "Where's the software?" This. This is why there is no software.
Modern software development practice is built around that humans do make mistakes. Frequently. You don't cross your fingers and hope for the best, you assume it will happen and you put processes in place to prevent errors, catch them early, and make them easy to recover from.
It is a for-real science.
There is no AI bro software? Claude code is clearly a vibe coded mess, and I would argue the most sucesfull dev software of the last 2 years.
Claude design eliminated figma & power point use for us.
We have a product out there as a test balloon, which only gets architecture & automated reviews.
Dev speed has been incredible, customers love it.
The mess that hey is, is also out there and not yet bankrupt with the biggest AI bro CEO out there
And I know at least 1 startup, which only hired a dev after having 20+ B2B customers. They sure do not go around advertising themself as vibe coded.
> And I know at least 1 startup, which only hired a dev after having 20+ B2B customers. They sure do not go around advertising themself as vibe coded.
Let me tell you something, I don't know jack about this new world.
But I do know about the old world. And in that old world, when you had 20 customers that actually needed your product for their business, your team of 5 people who built YOUR product are eagerly trying to talk to each one of them to figure out why your product works for them, how they found you, what they need now and in the future from your product. They reached out, took people to lunch, they went to meetups and hackathons and boring ass sales pitches to make connections.
Tell me about how all THAT stuff happens when your core dev team is agents in the cloud ?
1 reply →
Have the devs been financially rewarded for their speed or just given more work? Have the customers been charged less?
If "you don't have to review" really is true then the economics are an even bigger nightmare. The value of code has decreased, the "SaaS is dead" claim becomes real, and any good idea can be easily cloned.
> This attitude is exactly why AI bros don't make successful software lol.
There is the whole Artcraft thing.. We are now finally seeing the beginnings of successull fully vibe-coded software. This trend will only increase.
Have you actually used ArtCraft?
3 replies →
[dead]