Software works because someone is accountable for it. Pair Programming, Test Driven Development, and now Agentic coding all to one degree or another damage the network of trust.
- Pair Programming (There is no one person accountable for the change.)
- Test Driven Development (The test pass, so my software is good)
- Agentic Coding (My prompt was good, so my software is good)
AI is great for software development... As long as you review and understand each change it makes. Because YOU are going to be held accountable for what the software does.
I don't get this comment, you say it breaks trust because it's unknown who is accountable, and then you say you (the author) is accountable. So where is the trust broken?
Also your examples are wrong, regardless of pair programming or TDD, there is always a single committer, who is to be held accountable. How you achieve that commit in the past, copy-pasting from stackoverflow, or some auto-expand snippet automation from the IDE - so any tooling - does not matter. Agentic development is also such a tool, thus does not change trust.
The agent runner, or committer, a single human, is to be held accountable. That responsibility is - and always was - the most important role of a programmer.
How does opensource software work when nobody is accountable when I run software from GitHub?
This is the exact mentality behind corporate software and why nobody gets fired for using IBM, Oracle, Azure...
Accountability is a poor proxy for quality because there are so many ways to frame an incident never those responsible really get accountable.
Software works because developers care about the product. If something goes wrong the entire team is accountable, so they work together on something as one.
I am wondering if a combination of broader/deeper observability attach and synthetic user monitoring can help offset some of this. This may just displace and not mitigate the trust issue, but it seems like a valid product direction for the harness layer given falling token costs.
I started thinking about agentic programming from a totally different perspective.
Personal accountability in software development is actually hard, and less useful than it seems. What good is it if you discover a 12 y.o. vulnerability and can blame a specific person? (Like the recent one in PHP?) You may learn the name of that programmer who made the mistake, but it does not really tell you anything about that person, unless their overall record is terrible, and you are not entitled to any damages anyway.
In closed source, this is even worse. So a certain Windows function sucks. Now what. You never learn any other provenience than "Microsoft did it". Tough luck even getting support, and you certainly don't get your license fee back. This sort of accountability is worse than useless.
The only exception I see is knowledge within a specific development team. We usually know what sort of errors our colleagues are prone to, and we may be able to work around their personal weak spots.
Agentic coding done well has a dimension of trust that human coding cannot achieve. I can tell my agents to aim for 100 per cent test coverage without overburdening them or running out of budget and time. I can ask other agents and other models to review code written by the first agents, and then again. Every layer of such review adds some credibility to the result, and its mechanistic character ensures certain level of repeatability. Sure, it can happen that all involved mechanisms share the same weakness and let a particular bug through. But it is similar to several sieves over one another lining up perfectly to let something through, and that is quite rare.
If I had to choose, I would trust code designed, implemented and reviewed by three different SOTA models over code designed, implemented and reviewed by three median programmers today. Three elite programmers would have been a different story, but that would also be expensive like hell, and much like you don't eat most of your dinners in a Michelin restaurant, you have to economize with your code development in a similar way.
It's not about blaming a person, it's having a person or organization that has reputation to lose or gain and resources to fix it. Responsible AI coding usage increases both velocity and quality, but it's very easy to have issues become the responsibility of nobody.
> They are hesitant to take their hand off the wheel because they don’t have full confidence the agents have the right guardrails in place to run off on their own.
This is not a new problem, I've had this problem every summer when a new batch of interns show up.
As they say, if an intern can bring down production, it is not the intern's fault.
The interns are generally so much better than me at writing code, but they're not the best at "software development" (I call it "software gardening").
My training regime is to push them up the next 4 leadership levels from "writing code" - delegation, negotiation, vision and inspiration.
Agents are barely at delegation, with multi-agent harnesses.
They don't negotiate with you, they say yes.
Their vision is limited to the context you gave and most of their output is uninspiring (you can smell "Claude did this" on a UX design pattern easily).
But the humans, they've been thrust into delegation on day 1 & unless you've been a tech-lead manager, you probably haven't exercised that skill yet. Even if you have done that, the way agents work is not exactly how humans work [1] in terms of incentives or even try something they think won't work.
If you've been a good IC for years, the company could always afford to hire a human who specializes in those skills and suddenly you've been promoted to a new role without a pay bump or even a choice.
Most of the "Let's add agents and go faster" is basically adding more manpower to a project behind schedule and the payback is missing because that causes communication gaps & dropped messages.
A business ending software failure or security breach can occur from a single feature flag variable or 'minor' parsing bug. Usually its a long sequence of these, but it only takes one.
I've personally experienced week long incidents because someone didn't understand how a system worked and a couple of low impact shortcuts and convenient approaches and minor mistakes all came together to internally DDOS important systems. All reasonable individually, resulted in the majority of the ops team pulling a 70+ hour week and doing live prod cleanups in the middle of the night.... Agents do this faster. Also, they're labor competition, maybe.
This is why, to me, it's more interesting to ask the question "how are good devs attaining outsized successes with AI" than "why are good developers wary of AI?"
Yes, there are real reasons to be careful, but that's already obvious to everyone in the field. The real gold is in discovering how successful devs are managing these risks while also retaining the benefits.
This is also my experience in my own bubble. Even my friends who were skeptical for the longest started finding ways to perform cool cross-org work with them and accelerate their coding.
The biggest pocket of skilled holdouts I've noticed are in more artisan spaces where the craft of coding is part of the art. Things like old-school indie game developers where they're still sweating the assembly output and clock cycles.
> they are not hesitant to use a coding assistant, they are hesitant to go all in and let the agents do all the execution.
They are embracing the use of coding agents, but they still want to be in control. That also matches what I see from my most talented colleagues. They outsource certain tasks to the AI and verify the output.
My somewhat unpopular opinion is that only very good developers are able to use AI coding tools somewhat safely. They could have written the code themselves, but it's faster to have the agent do and then verify. The rest of us couldn't write the code, at least not when we start on a project. When we finish, we have some of the skills and could rely more on the AI tools if we were to do it again.
I have a suspicion that eventually you won't have to be a developer to successfully create good software. I do think that familiarity with algorithm design will still help a lot. But ultimately, if your verification, testing, and observability regimes are good enough, the exact implementation shouldn't matter.
The problems is the best devs aren’t the ones doing all the work. You have shitty devs too who are barely thinking or checking anything mixing slop code in with what whatever the best devs are doing, so it all eventually turns to shit.
The job of the best devs is to build the system ("the harness") that let's anyone build near their level of competency by unloading their heuristics into the harness. That is what the most senior, most talented devs on the team need to be doing so that the baseline gets lifted across the board.
Because LLM's can do amazing things, but they can also make huge blunders. Claude constantly makes incorrect assumptions, or implements the wrong thing, or claims there is a bug when there is not.....
On the other hand its VERY fast at finding and fixing some bugs, and finding problems before they hit production. Very good at checking things like what API changes an upgrade brings or finding the right answer about a badly documented library, etc.
That's a strawman (or at least a weakman): only a very small number of people (perhaps overrepresented online) thinks current agents are superhuman at programming in practice.
Which is of course not the same group as those who expect new models to become superhuman in months/years.
I think this blog post is underbaked. I think it misplaces where the tension is, and also the skiing metaphor is attempting to simplify an explanation that's already simple.
The post describes the tension as being between leaders and developers. I don't really think that's where the tension is. I'm on a small team of all senior devs, and I've seen both sides -- skepticism and full embrace, and leadership was not pushing AI on us at all. On our team, no matter where we're each individually at on adoption, we're all pretty clear-eyed about the upsides and downsides. The upside is absurdly fast code production. The downside is losing touch with every line of code in the codebase.
There's no need to introduce the skiing metaphor, because the problem is simple: it's the devs' necks on the line when the system goes down or a customer record goes missing. When you get a call at 4 in the morning from the CEO, "Let me ask someone (AI)" isn't an acceptable answer. It's your job to know the failure modes like the back of your hand and be able to diagnose and rightsize in minutes. We've spent our whole careers guarding against those failures by agonizing over every line of code that goes into the product. With that understanding, it's pretty obvious why senior devs are hesitant to adopt something new without a clear picture of how they can be sure that the program isn't going to go down under load, isn't going to drop or mangle customer records, and when one of those things inevitably does happen, how to recover quickly. In a nutshell, job #1 for a senior dev is to account for every failure, and this becomes much harder when an automated system is pushing through 10x as much code.
I'm one of the devs who has chosen to adopt AI. As I reflect on that, I think the reason I'm willing to do so is because I have a level of certainty that I can work the pipeline to safely introduce more changes, faster. So far, I am getting mixed results, but that's not deterring me from continuing to try to reach an optimal point on the tradeoff curve.
"agents" are useful for people operating in domains they don't know or understand, and not for people who do. Or at least I've never seen a task in a field I'm comfortable in, improved by either "AI" or "an AI agent". Made worse, yes, but not improved.
But I don't mind "agentic development" to help a person learn to code in a domain. As kind of a tutorial, both on "how things can work" and "how to spot the mistakes". "Agents" and "AI" always make a LOT of mistakes, especially in fields of domain specialization.
> How much can we trust a system when nobody has experience of building it this way?
If humans are no longer reading the LLM outputs and internalizing the codebase themselves, there's no longer much point to source code. The coding assistant might as well take a natural language prompt and directly hand you an ELF binary that passes a test suite. Are companies ready for that?
There do exist plenty of modest utility/novelty/supplementary projects for which that approach will probably prove fine and may even take over as a norm. I wouldn't make it my business to sell a product that could be built that way (where's the moat), but I'd consider using a product maintained that way as part of my business process. I'd just have its spec and maintenance be an internal matter overseen by an employee or contractor.
But for projects with expansive eat-the-world requirements and a need for extensive customization (enterprise and some large B2C platforms), or those with strict resource efficiency constraints, you can't just shake and bake a black box every time you have a new idea/requirement/bug. The search space to fit against is too vast, which even your model will need to pay for. You need to be working up from both a very detailed specification that can hold/verify all your constraints firm amidst changes and an architecture that sufficiently anticipated what you mean to add/fix.
Source code (as theory crafting) is really good at being the minimally-ambiguous documentation of those constraints and the analytical expression of that architecture. Where it matters, even in the context of agentic development, good source code will make your model get much more done for much less and so maintains your competitive advantage.
It's not going anywhere in the big picture, even if there will be more projects where it doesn't matter.
I think some definitely are. Especially if there is a system that sits on top of that and pokes and prods the binary to see if it behaves like it's supposed to.
Gary Tan has said markdown is code. Personally I see my job as establishing the architecture, creating tests, lint rules, etc. for my agents. Monitor the code, but if there's an issue create a new agent that monitors performance, etc.
Until spec files can be worked with so efficiently that we can just regenerate the binaries from them for every change, what you said makes little sense.
You're going to need an intermediate portable form of storing the program. It so happens that all the tools and AI are optimized to work with source code.
Exactly! I wouldn’t be surprised if some day in the distant future, that’s where this all goes. Coding languages may in fact be on their way to being dead. You can imagine a world where, regardless the platform, the lowest level instruction format is output.
How would we train those LLMs? Feed them binary executables along with their business requirement docs? Is the corpus of training data large enough to be effective? What happens when a new CPU architecture is developed? Where would the training data come from for the new machine code?
I can see a future generation of models. For now, LLM's train on human artifacts and they handle Python/TS well because of the overwhelming amount of code they've seen, combined with the semantic labels high level languages use that they rely on for reasoning.
Also, consider token efficiency. Imagine how many tokens would be torched just translating an import statement of a popular library to its binary equivalent, let alone a big complex program
Good developers know that AI isn't necessary at all and no software got better since AI generated code. In fact it got worse.
All that AI does is giving lazy developers more output that is approved by another LGTM developer, so the appearances of progress are kept up.
There is no reason at all to justify no-AI usage. It is the same as when the cigarette industry depicted smoking as cool and non-smokers had to justify not giving in to the dirty habit for fear of not being one of the cool kids.
This joke is made a lot but I think most good developers take care in considering the npm dependencies they install. You can get a ton of dependencies pretty easily by just installing very popular ones like react, tailwind, nextjs, etc. If there's ever a dependency that I'm not familiar with then I make sure to take at least a cursory look over the repo to make sure it has a decent number of stars/community, is well maintained, has multiple contributors, etc.
That is great, but the reality is that your OS, the productivity software you use daily, and the dependencies downstream of the ones you review, are not actually being checked by you for code quality.
Part of using software is just going off how well it works.
When one Linux flavor starts to have issues, most people don't do a dissection on why, they just switch distros. Or web browsers (I've lost count of the back-and-forth between FF and Chrome performance over the years, nevermind all their precursors). Or office suites.
Pretending that the code is what really matters, and not what the code does, is elevating the paint quality over the painting.
We need to build systems where trust in the code is not needed.
I don't trust the JavaScript code I run in my browser, I don't need to.
A language with a strong enough type system and effects system could say, "this function updates the users preferences, and may not access the network, and may not access data from other users", and the compiler can ensure these constraints AT COMPILE TIME! Provably correct code.
No doubt there's some hypocrisy to point to if you want, but most who say that kind of thing today have been saying the same thing about those "200 random npm dependencies" for the last decade too.
Where do you find these guys? I haven't worked on a project with more than 20 dependencies, and usually those dependencies have 0 subdependencies so they're quite easy to audit.
They're not. But some of them are! And it comes down to knowing how to work with agents in the best way. A good developer -- all in on Agentic development or not -- still knows precisely where to review, pause, rewrite, spike, and so on and so forth. I just don't see the problem here.
There's agentic development where you've designed, even at a high level what the goals and constraints are. Within reason, as doing 10 things at once is not compatible with actually thinking about a problem, no matter what your Adderall says.
Then there's... management forcing "agentic development" on you to basically hurry up, and actively don't want your design input or even to think about the problem. Just 'review it' and be expected to own the problems after. Outsourcing basically.
The first is fine, second is not sustainable for any happiness unless you are offering top compensation.
My experience as a senior software engineer is that generative AI is an incredible productivity boost for engineers that know what they need to implement. If I need to write a new service, I'm going to be able to do that extremely quickly with Claude Code or some other agentic tooling as long as I know what I want.
However, generative AI should be banned for leadership or product owners or anyone on the requirements specification end of the Jira ticket. I'm fine with them using LLMs to read documents and learn, but generative AI specifically in the hands of these folks seems to be a net productivity drag on organizations.
I am so tired of 1,500 word Jira tickets that are over-specified to death with requirements that nobody is actually interested in and I have to spend exorbitant amounts of times just trying to distill out the functionality that the requestor is actually looking for. And these logorrheic nightmare tickets end up spawning all kinds of other unnecessary downstream work.
Not wary really, but I do tend to micro manage the agent.
I'm no longer considering myself a good developer either, the agent is way better at details. Not so much at following a general pattern that makes sense.
> Good developers’ caution about agentic development comes from knowing that built isn’t the same as working in production...I always felt there was something else, another fear hiding behind those obvious ones.
It's not fear. It's experience. I've seen the kind of code agents write. And it's too often crap even when it "works". This isn't about beautiful code. It's about what we've built actually making sense to a, hopefully talented, software engineer.
Maybe more important than that, it's about taking the time to build the mental models that facilitate product development and the next iterations of coding. When agents write all the software, the majority of engineers I've seen, including myself, lose almost all context to what is actually going on at anything but a very high almost laughable level. And they then become almost completely dependent on the agent for the next iteration of development b/c they don't really understand what is there currently.
Does this matter? From a software perspective, it depends on what you are building and for whom.
From a human perspective, it depends on the human. I've seen good developers that I trust historically churn out crap code with agents because it was just too much to review. And there is some kind of weird addiction like thing happening in our brains when using AI that makes it incredibly difficult for us to slow down and review all the code it writes. In part b/c agents are great at crapping out 3K line PRs like it's nobody's business. But it needs to be someone's business to understand what we are shipping for anything but non-trivial projects. And truly understanding 3K line PRs hour after hour is just not sustainable for most of us. Our eyes glaze over and we just ship it because that's what we do now as an industry.
I think wary is the wrong term. I've learned agentic development's strengths and weaknesses. And the tools, as they work currently, churn out code at a rocket's pace that any good software engineer would immediately reject in years past. I don't like their code, I don't like how their code development process interacts with my brain. It's not waryness, it's dislike and mistrust.
So, yes, they can be assistants. And they can even write code. But I think in much smaller chunks than most people are using them these days.
The idea that we should build software factories and/or otherwise just turn agents loose and isolate humans to the product/QA level is insane to me. But I understand why we are here and I realize that a large portion of the business and dev community likes this new model.
I'm hoping that not doing it that way will end up be a competitive advantage in the long run. Both for the software we are producing and for the devs that are creating it.
> Maybe more important than that, it's about taking the time to build the mental models that facilitate product development and the next iterations of coding.
For me this is the big one. When you code by hand something (especially at the start of a project / feature), it gives you times and opportunity to evaluate different approach, like data-structure or overall architecture. When you're given agentic-generated code to review, those choices are already been made implicitly by Claude, and while it give you the impression that you understand the codebase, you actually haven't been able to process the tradeoff.
Of course, not 'hand-generated' code has the same ROI, and when you're about to do something derivative of something you already understand, agentic development is truly a huge productivity gain.
For me this is the difference between 'vibe coding' and 'coding with AI': Am I able to predict in advance the code that Claude is going to generate, and more importantly, am I able to be surprised when Claude doesn't generate the code I thought it would have (sometime for good technical reasons), or am I just looking at plausible-looking code diffs, and accepting thing without being able to understand the impact on the rest of the codebase. This for me is way more important than what percentage of code lines have been generated by AI.
I've had the opportunity to try both approach: one where I was given a vibecoded project and told to maintain it with Claude: I knew the details of some features, but haven't worked with the codebase enough to really understand the architecture, or be able to mentally reason about the codebase when I'm asked a question during a meeting.
On the another re-implementation of the same product, I've been given more leeway to develop how I wanted, so I've handcrafted the first few features in a transversal way (handling the file architecture, the routing, the way data and components are splitted, and how the API calls are handled), and then I've used Claude to generate the other feature (most of them I knew to be derivative of the first one). In the end something like 90% of the code is generated by Claude, but the end result is both much simpler because I knew where things should be simple and where edge case where, and most importantly I trust what I've put in production.
Do you review and understand every byte of machine code being generated by your compiler? Do you understand all the implications of how it could interact with the underlying hardware, pipelining and prediction, etc?
Maybe you do, but most of our industry doesn't and hasn't for decades. Isn't this just the next step up the ladder of abstraction?
Not OP, but the issue for me is that for the longest time doing software development meant translating potentially nebulous requirement into strict computer logic.
The process of creation is often a back-and forth between that fuzzy concept in your head (or an natural-language spec that is often ambiguous), and the (usually reduced) physical object / drawing / computer program you've created to materialize this idea. That imperfect projection of your initial idea then helps you refine what this idea implies, and what are the different tradeoff you could take [0].
That back-and-forth has to be made when writing the computer code, even if it's going to be compiled after, because code needs to be detailed enough to force you into expressing the desired behavior with clarity. Prompting Claude on the other hand usually means that you let the LLM resolve this ambiguity for you.
In some case the details don't matter, and it's a productivity gain, and in other case the devils are in the details, and you've let Claude make important technical decision for you, even though Claude doesn't know how much those decision match the needs of the project.
Personally I think of my role as a software developer as someone who know technical possibility (and technical limitations), and weight the different tradeoff it implies to propose code that solves a business problem. And that's the crux of the issue with only creating code by prompting Claude Code: it gradually robs me of my ability to understand the technical possibility and limitation of the system.
[0] This idea has largely been taken from the very good book "Sketching User Experiences: Getting the design right and the right design", about the processus of ideation / sketching.
And, for what I do, Python web apps, it's almost never a problem. It doesn't affect runtime performance, accuracy, maintainability, or bug frequency except in the rarest of cases. Finding non-determinism is even rarer. And, in almost all adverse cases, it would be considered a bug.
I therefore find your suggestion to compare machine code and LLM output questionable. I find them qualitatively different paradigms, not simply another step up in abstraction.
You can’t abstract away the need to capture requirements in a symbolic language. It doesn’t matter which programming or formal language you choose to specify your requirements in, but it cannot be natural language.
I'd push back on the use of the term "good developers" here. Since the beginning of software development, software developers have had to adapt to ever changing technologies. The good ones adapt, and the bad ones fall off. A person could say that they were great at writing assembly back in the 70s or 80s, but almost nobody writes straight assembly nowadays, and if they do it's usually written as inline assembly. The environment has once again changed, and agentic coding is the change. The actual good developers will use coding agents to complement their skills and write better code faster, while the ones who can't use or refuse to use coding agents will fall off.
Also the article mentions "senior engineers", the term "senior" is thrown around so much these days — along with the term "engineer" — that it's beginning to lose all meaning. Everyone is a senior after 2 years of software development it seems. Honestly if a group of "senior" "engineers" and infrastructure consultants couldn't get an application working in a week with agents behind their back, I would doubt their expertise. Maybe that airline application is super complicated, but it looks like they let the agents run hog wild without any oversight, which no shit they didn't get a working application in a week. Anyone who has used agents for coding knows you can't do that and expect good results for anything more than a toy application. Also I didn't see a mention of which agents they used, they are far from equal. They could be using Claude Haiku on low effort for all I know.
I think this experiment would have been more interesting with 3 teams: One team with no agents allowed, one team who do full on vibecoding like it seems in this article, and the last which uses agents to complement developers.
Nah, good developers are wary of agentic development because they have a high degree of understanding and intentionality for their designs. And they see how it easily becomes a crutch for a lack of understanding or intentionality.
> Agentic development will do to software what off-shoring did to manufacturing.
Agentic development will do to software what off-shoring did to software development.
Indeed, I think skills are like muscles... you need to keep using them well to stay strong. If you farm out all your thinking to an AI, your own skills and knowledge will diminish and your ability to design/debug/reason about your own (or any) code can fade.
as has happened time and again through history. software development skills will fade as models become genuinely better at creating, designing, architecting, engineering, testing, validating code at the level of fully complete products.
There will still be the need for humans in the loop to make sure the final product is suitable and what the humans genuinely wanted. We'll all become QA / Acceptance testers. For a little while , that will go away in the end too I think. software will become nothing more special than every day household objects.
For me it breaks the rule of "Network of trust".
Software works because someone is accountable for it. Pair Programming, Test Driven Development, and now Agentic coding all to one degree or another damage the network of trust.
- Pair Programming (There is no one person accountable for the change.) - Test Driven Development (The test pass, so my software is good) - Agentic Coding (My prompt was good, so my software is good)
AI is great for software development... As long as you review and understand each change it makes. Because YOU are going to be held accountable for what the software does.
Putting pair programming and TDD in the same box as agentic development (with, as you assume, zero review of the code) seems...
I don't get this comment, you say it breaks trust because it's unknown who is accountable, and then you say you (the author) is accountable. So where is the trust broken?
Also your examples are wrong, regardless of pair programming or TDD, there is always a single committer, who is to be held accountable. How you achieve that commit in the past, copy-pasting from stackoverflow, or some auto-expand snippet automation from the IDE - so any tooling - does not matter. Agentic development is also such a tool, thus does not change trust.
The agent runner, or committer, a single human, is to be held accountable. That responsibility is - and always was - the most important role of a programmer.
How does opensource software work when nobody is accountable when I run software from GitHub?
This is the exact mentality behind corporate software and why nobody gets fired for using IBM, Oracle, Azure...
Accountability is a poor proxy for quality because there are so many ways to frame an incident never those responsible really get accountable.
Software works because developers care about the product. If something goes wrong the entire team is accountable, so they work together on something as one.
Open source projects themselves have reputation, regardless of how they are governed specifically.
I don't think pairing, nor TDD come anywhere close to the abdication of responsibility that agentic programming does.
I am wondering if a combination of broader/deeper observability attach and synthetic user monitoring can help offset some of this. This may just displace and not mitigate the trust issue, but it seems like a valid product direction for the harness layer given falling token costs.
I started thinking about agentic programming from a totally different perspective.
Personal accountability in software development is actually hard, and less useful than it seems. What good is it if you discover a 12 y.o. vulnerability and can blame a specific person? (Like the recent one in PHP?) You may learn the name of that programmer who made the mistake, but it does not really tell you anything about that person, unless their overall record is terrible, and you are not entitled to any damages anyway.
In closed source, this is even worse. So a certain Windows function sucks. Now what. You never learn any other provenience than "Microsoft did it". Tough luck even getting support, and you certainly don't get your license fee back. This sort of accountability is worse than useless.
The only exception I see is knowledge within a specific development team. We usually know what sort of errors our colleagues are prone to, and we may be able to work around their personal weak spots.
Agentic coding done well has a dimension of trust that human coding cannot achieve. I can tell my agents to aim for 100 per cent test coverage without overburdening them or running out of budget and time. I can ask other agents and other models to review code written by the first agents, and then again. Every layer of such review adds some credibility to the result, and its mechanistic character ensures certain level of repeatability. Sure, it can happen that all involved mechanisms share the same weakness and let a particular bug through. But it is similar to several sieves over one another lining up perfectly to let something through, and that is quite rare.
If I had to choose, I would trust code designed, implemented and reviewed by three different SOTA models over code designed, implemented and reviewed by three median programmers today. Three elite programmers would have been a different story, but that would also be expensive like hell, and much like you don't eat most of your dinners in a Michelin restaurant, you have to economize with your code development in a similar way.
It's not about blaming a person, it's having a person or organization that has reputation to lose or gain and resources to fix it. Responsible AI coding usage increases both velocity and quality, but it's very easy to have issues become the responsibility of nobody.
> They are hesitant to take their hand off the wheel because they don’t have full confidence the agents have the right guardrails in place to run off on their own.
This is not a new problem, I've had this problem every summer when a new batch of interns show up.
As they say, if an intern can bring down production, it is not the intern's fault.
The interns are generally so much better than me at writing code, but they're not the best at "software development" (I call it "software gardening").
My training regime is to push them up the next 4 leadership levels from "writing code" - delegation, negotiation, vision and inspiration.
Agents are barely at delegation, with multi-agent harnesses.
They don't negotiate with you, they say yes.
Their vision is limited to the context you gave and most of their output is uninspiring (you can smell "Claude did this" on a UX design pattern easily).
But the humans, they've been thrust into delegation on day 1 & unless you've been a tech-lead manager, you probably haven't exercised that skill yet. Even if you have done that, the way agents work is not exactly how humans work [1] in terms of incentives or even try something they think won't work.
If you've been a good IC for years, the company could always afford to hire a human who specializes in those skills and suddenly you've been promoted to a new role without a pay bump or even a choice.
Most of the "Let's add agents and go faster" is basically adding more manpower to a project behind schedule and the payback is missing because that causes communication gaps & dropped messages.
[1] - https://tinyurl.com/orchestration-imperative
A business ending software failure or security breach can occur from a single feature flag variable or 'minor' parsing bug. Usually its a long sequence of these, but it only takes one.
I've personally experienced week long incidents because someone didn't understand how a system worked and a couple of low impact shortcuts and convenient approaches and minor mistakes all came together to internally DDOS important systems. All reasonable individually, resulted in the majority of the ops team pulling a 70+ hour week and doing live prod cleanups in the middle of the night.... Agents do this faster. Also, they're labor competition, maybe.
I’ve seen the opposite. The best devs are aggressively embracing agents - producing more, and at higher quality.
This is why, to me, it's more interesting to ask the question "how are good devs attaining outsized successes with AI" than "why are good developers wary of AI?"
Yes, there are real reasons to be careful, but that's already obvious to everyone in the field. The real gold is in discovering how successful devs are managing these risks while also retaining the benefits.
This is also my experience in my own bubble. Even my friends who were skeptical for the longest started finding ways to perform cool cross-org work with them and accelerate their coding.
The biggest pocket of skilled holdouts I've noticed are in more artisan spaces where the craft of coding is part of the art. Things like old-school indie game developers where they're still sweating the assembly output and clock cycles.
From the article:
> they are not hesitant to use a coding assistant, they are hesitant to go all in and let the agents do all the execution.
They are embracing the use of coding agents, but they still want to be in control. That also matches what I see from my most talented colleagues. They outsource certain tasks to the AI and verify the output.
My somewhat unpopular opinion is that only very good developers are able to use AI coding tools somewhat safely. They could have written the code themselves, but it's faster to have the agent do and then verify. The rest of us couldn't write the code, at least not when we start on a project. When we finish, we have some of the skills and could rely more on the AI tools if we were to do it again.
I have a suspicion that eventually you won't have to be a developer to successfully create good software. I do think that familiarity with algorithm design will still help a lot. But ultimately, if your verification, testing, and observability regimes are good enough, the exact implementation shouldn't matter.
1 reply →
Yes sir! 100 times this.
The problems is the best devs aren’t the ones doing all the work. You have shitty devs too who are barely thinking or checking anything mixing slop code in with what whatever the best devs are doing, so it all eventually turns to shit.
The job of the best devs is to build the system ("the harness") that let's anyone build near their level of competency by unloading their heuristics into the harness. That is what the most senior, most talented devs on the team need to be doing so that the baseline gets lifted across the board.
It won't be long before the best devs are working and the shit ones are not
2 replies →
Because LLM's can do amazing things, but they can also make huge blunders. Claude constantly makes incorrect assumptions, or implements the wrong thing, or claims there is a bug when there is not.....
On the other hand its VERY fast at finding and fixing some bugs, and finding problems before they hit production. Very good at checking things like what API changes an upgrade brings or finding the right answer about a badly documented library, etc.
It's almost like LLMs are another tool with strengths and weaknesses rather than a super human intelligence.
Its exactly what it is, a tool. Use it correctly it works fine but if you are not careful you can shoot yourself in the foot.
> rather than a super human intelligence
That's a strawman (or at least a weakman): only a very small number of people (perhaps overrepresented online) thinks current agents are superhuman at programming in practice.
Which is of course not the same group as those who expect new models to become superhuman in months/years.
1 reply →
I think this blog post is underbaked. I think it misplaces where the tension is, and also the skiing metaphor is attempting to simplify an explanation that's already simple.
The post describes the tension as being between leaders and developers. I don't really think that's where the tension is. I'm on a small team of all senior devs, and I've seen both sides -- skepticism and full embrace, and leadership was not pushing AI on us at all. On our team, no matter where we're each individually at on adoption, we're all pretty clear-eyed about the upsides and downsides. The upside is absurdly fast code production. The downside is losing touch with every line of code in the codebase.
There's no need to introduce the skiing metaphor, because the problem is simple: it's the devs' necks on the line when the system goes down or a customer record goes missing. When you get a call at 4 in the morning from the CEO, "Let me ask someone (AI)" isn't an acceptable answer. It's your job to know the failure modes like the back of your hand and be able to diagnose and rightsize in minutes. We've spent our whole careers guarding against those failures by agonizing over every line of code that goes into the product. With that understanding, it's pretty obvious why senior devs are hesitant to adopt something new without a clear picture of how they can be sure that the program isn't going to go down under load, isn't going to drop or mangle customer records, and when one of those things inevitably does happen, how to recover quickly. In a nutshell, job #1 for a senior dev is to account for every failure, and this becomes much harder when an automated system is pushing through 10x as much code.
I'm one of the devs who has chosen to adopt AI. As I reflect on that, I think the reason I'm willing to do so is because I have a level of certainty that I can work the pipeline to safely introduce more changes, faster. So far, I am getting mixed results, but that's not deterring me from continuing to try to reach an optimal point on the tradeoff curve.
"agents" are useful for people operating in domains they don't know or understand, and not for people who do. Or at least I've never seen a task in a field I'm comfortable in, improved by either "AI" or "an AI agent". Made worse, yes, but not improved.
But I don't mind "agentic development" to help a person learn to code in a domain. As kind of a tutorial, both on "how things can work" and "how to spot the mistakes". "Agents" and "AI" always make a LOT of mistakes, especially in fields of domain specialization.
They are still hanging into the false belief that software quality matters to anyone besides good developers.
You don't get frustrated when things that should work don't?
My frustration doesn't matter to anyone except me.
After spending a few hours looking at some human written C with daybreak I no longer trust human made code. It's mostly slop.
> How much can we trust a system when nobody has experience of building it this way?
If humans are no longer reading the LLM outputs and internalizing the codebase themselves, there's no longer much point to source code. The coding assistant might as well take a natural language prompt and directly hand you an ELF binary that passes a test suite. Are companies ready for that?
There is a point to source code:
Lots of tools for analysing and debugging it
It can be cross platform - the same source code will run on x86 and ARM, and in many cases on Windows and Linux.
Changes have predictable consequences because compilers and interpreters are deterministic.
There do exist plenty of modest utility/novelty/supplementary projects for which that approach will probably prove fine and may even take over as a norm. I wouldn't make it my business to sell a product that could be built that way (where's the moat), but I'd consider using a product maintained that way as part of my business process. I'd just have its spec and maintenance be an internal matter overseen by an employee or contractor.
But for projects with expansive eat-the-world requirements and a need for extensive customization (enterprise and some large B2C platforms), or those with strict resource efficiency constraints, you can't just shake and bake a black box every time you have a new idea/requirement/bug. The search space to fit against is too vast, which even your model will need to pay for. You need to be working up from both a very detailed specification that can hold/verify all your constraints firm amidst changes and an architecture that sufficiently anticipated what you mean to add/fix.
Source code (as theory crafting) is really good at being the minimally-ambiguous documentation of those constraints and the analytical expression of that architecture. Where it matters, even in the context of agentic development, good source code will make your model get much more done for much less and so maintains your competitive advantage.
It's not going anywhere in the big picture, even if there will be more projects where it doesn't matter.
I think some definitely are. Especially if there is a system that sits on top of that and pokes and prods the binary to see if it behaves like it's supposed to.
Gary Tan has said markdown is code. Personally I see my job as establishing the architecture, creating tests, lint rules, etc. for my agents. Monitor the code, but if there's an issue create a new agent that monitors performance, etc.
That's simply not true.
Until spec files can be worked with so efficiently that we can just regenerate the binaries from them for every change, what you said makes little sense.
You're going to need an intermediate portable form of storing the program. It so happens that all the tools and AI are optimized to work with source code.
Exactly, at what point do spec files become so precise that they are just another form of source code?
1 reply →
Exactly! I wouldn’t be surprised if some day in the distant future, that’s where this all goes. Coding languages may in fact be on their way to being dead. You can imagine a world where, regardless the platform, the lowest level instruction format is output.
How would we train those LLMs? Feed them binary executables along with their business requirement docs? Is the corpus of training data large enough to be effective? What happens when a new CPU architecture is developed? Where would the training data come from for the new machine code?
I can see a future generation of models. For now, LLM's train on human artifacts and they handle Python/TS well because of the overwhelming amount of code they've seen, combined with the semantic labels high level languages use that they rely on for reasoning.
Also, consider token efficiency. Imagine how many tokens would be torched just translating an import statement of a popular library to its binary equivalent, let alone a big complex program
I would not be surprised if we ended up with binaries and Lean proofs.
isnt that kinda like downloading and using software a team you have no knowledge of has produced and released?
All you can go by is 'does it do what it says it does'
Yeah, all these devs pretending you can only trust code you understand, then going and pulling in 1000 random npm deps...
2 replies →
I feel like ever since we stopped reading assembly we don't really internalize our codebases.
It's just one "Buy Now" button. How hard could that be? /s
Because they are stubborn.
When even Linus is using agents it is time to accept it.
[dead]
[dead]
Good developers know that AI isn't necessary at all and no software got better since AI generated code. In fact it got worse.
All that AI does is giving lazy developers more output that is approved by another LGTM developer, so the appearances of progress are kept up.
There is no reason at all to justify no-AI usage. It is the same as when the cigarette industry depicted smoking as cool and non-smokers had to justify not giving in to the dirty habit for fear of not being one of the cool kids.
"I don't trust code that I don't understand, or that I don't know someone experienced has reviewed."
proceeds to install 200 random npm dependencies
This joke is made a lot but I think most good developers take care in considering the npm dependencies they install. You can get a ton of dependencies pretty easily by just installing very popular ones like react, tailwind, nextjs, etc. If there's ever a dependency that I'm not familiar with then I make sure to take at least a cursory look over the repo to make sure it has a decent number of stars/community, is well maintained, has multiple contributors, etc.
That is great, but the reality is that your OS, the productivity software you use daily, and the dependencies downstream of the ones you review, are not actually being checked by you for code quality.
Part of using software is just going off how well it works.
When one Linux flavor starts to have issues, most people don't do a dissection on why, they just switch distros. Or web browsers (I've lost count of the back-and-forth between FF and Chrome performance over the years, nevermind all their precursors). Or office suites.
Pretending that the code is what really matters, and not what the code does, is elevating the paint quality over the painting.
3 replies →
>> I think most good developers take care in considering the npm dependencies they install
Evidence?!
No they don’t - developers just fly past installing whatever they need to make the damn thing work.
They don’t sit there thinking carefully about what each dependency means.
I don't think this is the class of developer the article is referencing
We need to build systems where trust in the code is not needed.
I don't trust the JavaScript code I run in my browser, I don't need to.
A language with a strong enough type system and effects system could say, "this function updates the users preferences, and may not access the network, and may not access data from other users", and the compiler can ensure these constraints AT COMPILE TIME! Provably correct code.
Who knew it's possible to simply have perfect vuln-free code!
We security engineers are definitely shaking in our boots.
No doubt there's some hypocrisy to point to if you want, but most who say that kind of thing today have been saying the same thing about those "200 random npm dependencies" for the last decade too.
Probably not the same people.
The good devs I know avoid dependencies like the plague.
Where do you find these guys? I haven't worked on a project with more than 20 dependencies, and usually those dependencies have 0 subdependencies so they're quite easy to audit.
Then you probably haven't worked much with JS.
The Jest test runner alone has 250 dependencies in its latest version, and v25 had over 500.
Are you including transient dependencies in that count?
3 replies →
They're not. But some of them are! And it comes down to knowing how to work with agents in the best way. A good developer -- all in on Agentic development or not -- still knows precisely where to review, pause, rewrite, spike, and so on and so forth. I just don't see the problem here.
I’m weary of the culture of ship it if it passes tests even if someone (a reviewer) has concerns
There's agentic development where you've designed, even at a high level what the goals and constraints are. Within reason, as doing 10 things at once is not compatible with actually thinking about a problem, no matter what your Adderall says.
Then there's... management forcing "agentic development" on you to basically hurry up, and actively don't want your design input or even to think about the problem. Just 'review it' and be expected to own the problems after. Outsourcing basically.
The first is fine, second is not sustainable for any happiness unless you are offering top compensation.
My experience as a senior software engineer is that generative AI is an incredible productivity boost for engineers that know what they need to implement. If I need to write a new service, I'm going to be able to do that extremely quickly with Claude Code or some other agentic tooling as long as I know what I want.
However, generative AI should be banned for leadership or product owners or anyone on the requirements specification end of the Jira ticket. I'm fine with them using LLMs to read documents and learn, but generative AI specifically in the hands of these folks seems to be a net productivity drag on organizations.
I am so tired of 1,500 word Jira tickets that are over-specified to death with requirements that nobody is actually interested in and I have to spend exorbitant amounts of times just trying to distill out the functionality that the requestor is actually looking for. And these logorrheic nightmare tickets end up spawning all kinds of other unnecessary downstream work.
Not wary really, but I do tend to micro manage the agent.
I'm no longer considering myself a good developer either, the agent is way better at details. Not so much at following a general pattern that makes sense.
Just because you’re not a good developer, doesn’t mean you can’t be a “good” developer.
[dead]
not sure whether I am good developer but for me it is half of
> The easy (lazy?) answer is that they love writing code, or they’re worried about their jobs.
and half of using proprietary SaaS being something I deeply dislike.
> Good developers’ caution about agentic development comes from knowing that built isn’t the same as working in production...I always felt there was something else, another fear hiding behind those obvious ones.
It's not fear. It's experience. I've seen the kind of code agents write. And it's too often crap even when it "works". This isn't about beautiful code. It's about what we've built actually making sense to a, hopefully talented, software engineer.
Maybe more important than that, it's about taking the time to build the mental models that facilitate product development and the next iterations of coding. When agents write all the software, the majority of engineers I've seen, including myself, lose almost all context to what is actually going on at anything but a very high almost laughable level. And they then become almost completely dependent on the agent for the next iteration of development b/c they don't really understand what is there currently.
Does this matter? From a software perspective, it depends on what you are building and for whom.
From a human perspective, it depends on the human. I've seen good developers that I trust historically churn out crap code with agents because it was just too much to review. And there is some kind of weird addiction like thing happening in our brains when using AI that makes it incredibly difficult for us to slow down and review all the code it writes. In part b/c agents are great at crapping out 3K line PRs like it's nobody's business. But it needs to be someone's business to understand what we are shipping for anything but non-trivial projects. And truly understanding 3K line PRs hour after hour is just not sustainable for most of us. Our eyes glaze over and we just ship it because that's what we do now as an industry.
I think wary is the wrong term. I've learned agentic development's strengths and weaknesses. And the tools, as they work currently, churn out code at a rocket's pace that any good software engineer would immediately reject in years past. I don't like their code, I don't like how their code development process interacts with my brain. It's not waryness, it's dislike and mistrust.
So, yes, they can be assistants. And they can even write code. But I think in much smaller chunks than most people are using them these days.
The idea that we should build software factories and/or otherwise just turn agents loose and isolate humans to the product/QA level is insane to me. But I understand why we are here and I realize that a large portion of the business and dev community likes this new model.
I'm hoping that not doing it that way will end up be a competitive advantage in the long run. Both for the software we are producing and for the devs that are creating it.
> Maybe more important than that, it's about taking the time to build the mental models that facilitate product development and the next iterations of coding.
For me this is the big one. When you code by hand something (especially at the start of a project / feature), it gives you times and opportunity to evaluate different approach, like data-structure or overall architecture. When you're given agentic-generated code to review, those choices are already been made implicitly by Claude, and while it give you the impression that you understand the codebase, you actually haven't been able to process the tradeoff.
Of course, not 'hand-generated' code has the same ROI, and when you're about to do something derivative of something you already understand, agentic development is truly a huge productivity gain.
For me this is the difference between 'vibe coding' and 'coding with AI': Am I able to predict in advance the code that Claude is going to generate, and more importantly, am I able to be surprised when Claude doesn't generate the code I thought it would have (sometime for good technical reasons), or am I just looking at plausible-looking code diffs, and accepting thing without being able to understand the impact on the rest of the codebase. This for me is way more important than what percentage of code lines have been generated by AI.
I've had the opportunity to try both approach: one where I was given a vibecoded project and told to maintain it with Claude: I knew the details of some features, but haven't worked with the codebase enough to really understand the architecture, or be able to mentally reason about the codebase when I'm asked a question during a meeting.
On the another re-implementation of the same product, I've been given more leeway to develop how I wanted, so I've handcrafted the first few features in a transversal way (handling the file architecture, the routing, the way data and components are splitted, and how the API calls are handled), and then I've used Claude to generate the other feature (most of them I knew to be derivative of the first one). In the end something like 90% of the code is generated by Claude, but the end result is both much simpler because I knew where things should be simple and where edge case where, and most importantly I trust what I've put in production.
Do you review and understand every byte of machine code being generated by your compiler? Do you understand all the implications of how it could interact with the underlying hardware, pipelining and prediction, etc?
Maybe you do, but most of our industry doesn't and hasn't for decades. Isn't this just the next step up the ladder of abstraction?
Not OP, but the issue for me is that for the longest time doing software development meant translating potentially nebulous requirement into strict computer logic.
The process of creation is often a back-and forth between that fuzzy concept in your head (or an natural-language spec that is often ambiguous), and the (usually reduced) physical object / drawing / computer program you've created to materialize this idea. That imperfect projection of your initial idea then helps you refine what this idea implies, and what are the different tradeoff you could take [0].
That back-and-forth has to be made when writing the computer code, even if it's going to be compiled after, because code needs to be detailed enough to force you into expressing the desired behavior with clarity. Prompting Claude on the other hand usually means that you let the LLM resolve this ambiguity for you.
In some case the details don't matter, and it's a productivity gain, and in other case the devils are in the details, and you've let Claude make important technical decision for you, even though Claude doesn't know how much those decision match the needs of the project.
Personally I think of my role as a software developer as someone who know technical possibility (and technical limitations), and weight the different tradeoff it implies to propose code that solves a business problem. And that's the crux of the issue with only creating code by prompting Claude Code: it gradually robs me of my ability to understand the technical possibility and limitation of the system.
[0] This idea has largely been taken from the very good book "Sketching User Experiences: Getting the design right and the right design", about the processus of ideation / sketching.
No, I don't.
And, for what I do, Python web apps, it's almost never a problem. It doesn't affect runtime performance, accuracy, maintainability, or bug frequency except in the rarest of cases. Finding non-determinism is even rarer. And, in almost all adverse cases, it would be considered a bug.
I therefore find your suggestion to compare machine code and LLM output questionable. I find them qualitatively different paradigms, not simply another step up in abstraction.
You can’t abstract away the need to capture requirements in a symbolic language. It doesn’t matter which programming or formal language you choose to specify your requirements in, but it cannot be natural language.
I'd push back on the use of the term "good developers" here. Since the beginning of software development, software developers have had to adapt to ever changing technologies. The good ones adapt, and the bad ones fall off. A person could say that they were great at writing assembly back in the 70s or 80s, but almost nobody writes straight assembly nowadays, and if they do it's usually written as inline assembly. The environment has once again changed, and agentic coding is the change. The actual good developers will use coding agents to complement their skills and write better code faster, while the ones who can't use or refuse to use coding agents will fall off.
Also the article mentions "senior engineers", the term "senior" is thrown around so much these days — along with the term "engineer" — that it's beginning to lose all meaning. Everyone is a senior after 2 years of software development it seems. Honestly if a group of "senior" "engineers" and infrastructure consultants couldn't get an application working in a week with agents behind their back, I would doubt their expertise. Maybe that airline application is super complicated, but it looks like they let the agents run hog wild without any oversight, which no shit they didn't get a working application in a week. Anyone who has used agents for coding knows you can't do that and expect good results for anything more than a toy application. Also I didn't see a mention of which agents they used, they are far from equal. They could be using Claude Haiku on low effort for all I know.
I think this experiment would have been more interesting with 3 teams: One team with no agents allowed, one team who do full on vibecoding like it seems in this article, and the last which uses agents to complement developers.
>> TL;DR Good developers’ caution about agentic development comes from knowing that built isn’t the same as working in production.
No it’s not about some superior knowledge of how things work in the real world.
Seems to me it’s mostly emotional reaction not logical, which is fine.
The important thing to understand is that emotional reaction won’t change when the tech improves - they’ll still hate AI.
Why are good developers so wary of agentic development?
Perhaps because agentic development is aimed at replacing "good developers".
Agentic development tends to undermine the very process that creates "good developers".
Agentic development will do to software what off-shoring did to manufacturing.
Nah, good developers are wary of agentic development because they have a high degree of understanding and intentionality for their designs. And they see how it easily becomes a crutch for a lack of understanding or intentionality.
> Agentic development will do to software what off-shoring did to manufacturing.
Agentic development will do to software what off-shoring did to software development.
And they see how it easily becomes a crutch for a lack of understanding or intentionality.
In other words, it "tends to undermine the very process that creates good developers".
Indeed, I think skills are like muscles... you need to keep using them well to stay strong. If you farm out all your thinking to an AI, your own skills and knowledge will diminish and your ability to design/debug/reason about your own (or any) code can fade.
as has happened time and again through history. software development skills will fade as models become genuinely better at creating, designing, architecting, engineering, testing, validating code at the level of fully complete products.
There will still be the need for humans in the loop to make sure the final product is suitable and what the humans genuinely wanted. We'll all become QA / Acceptance testers. For a little while , that will go away in the end too I think. software will become nothing more special than every day household objects.
3 replies →
[flagged]
[dead]