Comment by rglover
19 days ago
A significant number of developers and businesses are going to have an absolutely brutal rude awakening in the not too distant future.
You can build things this way, and they may work for a time, but you don't know what you don't know (and experience teaches you that you only find most stuff by building/struggling; not sipping a soda while the AI blurts out potentially secure/stable code).
The hubris around AI is going to be hard to watch unwind. What the moment is I can't predict (nor do I care to), but there will be a shift when all of these vibe code only folks get cooked in a way that's closer to existential than benign.
Good time to be in business if you can see through the bs and understand how these systems actually function (hint: you won't have much competition soon as most people won't care until it's too late and will "price themselves out of the market").
I would argue that it's going to be the opposite. At re:Invent, one of the popular sessions was in creating a trio of SRE agents, one of which did nothing but read logs and report errors, one of which did analysis of the errors and triaged and proposed fixes, and one to do the work and submit PRs to your repo.
Then, as part of the session, you would artificially introduce a bug into the system, then run into the bug in your browser. You'd see the failure happen in browser, and looking at Cloudwatch logs you'd see the error get logged.
Two minutes later, the SRE agents had the bug fixed and ready to be merged.
"understand how these systems actually function" isn't incompatible with "I didn't write most of this code". Unless you are only ever a single engineer, your career is filled with "I need to debug code I didn't write". What we have seen over the past few months is a gigantic leap in output quality, such that re-prompting happens less and less. Additionally, "after you've written this, document the logic within this markdown file" is extremely useful for your own reference and for future LLM sessions.
AWS is making a huge, huge bet on this being the future of software engineering, and even though they have their weird AWS-ish lock-in for some of the LLM-adjacent practices, it is an extremely compelling vision, and as these nondeterministic tools get more deterministic supporting functions to help their work, the quality is going to approach and probably exceed human coding quality.
I agree with both you and the GP. Yes, coding is being totally revolutionized by AI, and we don't really know where the ceiling will be (though I'm skeptical we'll reach true AGI any time soon), but I believe there still an essential element of understanding how computer systems work that is required to leverage AI in a sustainable way.
There is some combination of curiosity of inner workings and precision of thought that has always been essential in becoming a successful engineer. In my very first CS 101 class I remember the professor alluding to two hurdles (pointers and recursion) which a significant portion of the class would not be able to surpass and they would change majors. Throughout the subsequent decades I saw this pattern again and again with junior engineers, bootcamp grads, etc. There are some people no matter how hard they work, they can't grok abstraction and unlock a general understanding of computing possibility.
With AI you don't need to know syntax anymore, but to write the write prompts to maintain a system and (crucially) the integrity of its data over time, you still need this understanding. I'm not sure how the AI-native generation of software engineers will develop this without writing code hands-on, but I am confident they will figure it out because I believe it to be an innate, often pedantic, thirst for understanding that some people have and some don't. This is the essential quality to succeed in software both in the past and in the future. Although vibe coding lowers the barrier to entry dramatically, there is a brick wall looming just beyond the toy app/prototype phase for anyone without a technical mindset.
I can see why people are skeptical devs can be 10x as productive.
But something I'd bet money on is that devs are 10x more productive at using these tools.
44 replies →
It is tempting to think that we can delegate describing the mental model to AI, but it seems like all of this boils down to humans making bets, and it also seems like the fundamental bets engineers are making are about the formalisms that encode the product and make it valuable.
What an awful professor! When I first tried to learn pointers, I didn't get it. I tried again 6 months later and suddenly it clicked. The same thing happened for another guy I was learning with.
So the professor just gaslit years of students into thinking they were too dumb to get programming, and also left them with the developmental disability of "if you can't figure something out in a few days, you'll never get it".
1 reply →
I don’t think there will be an “AI native” generation of developers. AI will be the entity that “groks pointers” and no one else will know it or care what goes on under the hood.
Speaking as someone who has been both a SRE/DevOps from all levels from IC to Global Head of a team:
- I 100% believe this is happening and is probably going to be the case in the next 6 months. I've seen Claude and Grok debug issues when they only had half of the relevant evidence (e.g. Given A and B, it's most likely X). It can even debug complex issues between systems using logs, metrics etc. In other words, everything a human would do (and sometimes better).
- The situation described is actually not that different from being a SRE manager. e.g. as you get more senior, you aren't doing the investigations yourself. It's usually your direct reports that are actually looking at the logs etc. You may occasionally get involved for more complex issues or big outages but the direct reports are doing a lot of the heavy lifting.
- All of the above being said, I can imagine errors so weird/complex etc that the LLMs either can't figure it out, don't have the MCP or skill to resolve it or there is some giant technology issue that breaks a lot of stuff. Facebook engineers using angle grinders to get into the data center due to DNS issues comes to mind for the last one.
Which probably means we are all going to start to be more like airline pilots:
- highly trained in debugging AND managing fleets of LLMs
- managing autonomous systems
- around "just in case" the LLMs fall over
P.S. I've been very well paid over the years and being a SRE is how I feed my family. I do worry, like many, about how all of this is going to affect that. Sobering stuff.
-
> Which probably means we are all going to start to be more like airline pilots:
Airline pilots are still employed because of regulations. The industry is heavily regulated and the regulations move very slowly because of its international cooperative nature. The regulations dictate how many crew members should be on board for each plane type and other various variables. All the airlines have to abide by the rules of the airspace they're flying over to keep flying.
The airlines on the other hand along with the technology producers (airbus for example) are pursuing to reduce number of heads in the cockpit. While their recent attempt to get rid of co-pilots in EASA land has failed [1], you can see the amount of pursuit and investment. The industry will continue to force through cost optimization as long as there's no barrier to prevent. The cases where automation has failed will be just a cost of the business, since the life of the deceased is no concern to the company's balance sheet.
Given the lack of regulation in the software, I suspect the industry will continue the cost optimization and eliminate humans in the loop, except in the regulated domains.
[1] - https://www.easa.europa.eu/en/research-projects/emco-sipo-ex... ; while this was not a direct push to get rid of all pilots, it's a stepping stone in that direction.
3 replies →
Now run that loop 1000 times.
What does the code /system look like.
It is going to be more like evolution (fit to environment) than engineering (fit to purpose).
It will be fascinating to watch nonetheless.
It'll probably look like the code version of this, an image run through a LLM 101 times with the directive to create a replica of the input image: https://www.reddit.com/r/ChatGPT/comments/1kbj71z/i_tried_th... Despite being provided with explicit instructions, well...
People are still wrongly attributing a mind to something that is essentially mindless.
7 replies →
"evolution (fit to environment) than engineering (fit to purpose)."
Oh, I absolutely love this lens.
Sure, if all you ask it to do is fix bugs. You can also ask it to work on code health things like better organization, better testing, finding interesting invariants and enforcing them, and so on.
It's up to you what you want to prioritize.
18 replies →
> Unless you are only ever a single engineer, your career is filled with "I need to debug code I didn't write".
That's the vast majority of my job and I've yet to find a way to have LLMs not be almost but not entirely useless at helping me with it.
(also, it's filled with that even when you are a single engineer)
And even if you are the single engineer, I'll be honest, it might as well have been somebody else that wrote the code if I have to go back to something I did seven years ago and unearth wtf.
I hope you realize that means your position is in danger.
3 replies →
It's nice that AI can fix bugs fast, but it's better to not even have bugs in the first place. By using someone else's battle tested code (like a framework) you can at least avoid the bugs they've already encountered and fixed.
I spent Dry January working on a new coding project and since all my nerd friends have been telling me to try to code with LLM's I gave it a shot and signed up to Google Gemini...
All I can say is "holy shit, I'm a believer." I've probably got close to a year's worth of coding done in a month and a half.
Busy work that would have taken me a day to look up, figure out, and write -- boring shit like matplotlib illustrations -- they are trivial now.
Things that are ideas that I'm not sure how to implement "what are some different ways to do this weird thing" that I would have spend a week on trying to figure out a reasonable approach, no, it's basically got two or three decent ideas right away, even if they're not perfect. There was one vectorization approach I would have never thought of that I'm now using.
Is the LLM wrong? Yes, all the damn time! Do I need to, you know, actually do a code review then I'm implementing ideas? Very much yes! Do I get into a back and forth battle with the LLM when it gets starts spitting out nonsense, shut the chat down, and start over with a newly primed window? Yes, about once every couple of days.
It's still absolutely incredible. I've been a skeptic for a very long time. I studied philosophy, and the conceptions people have of language and Truth get completely garbled by an LLM that isn't really a mind that can think in the way we do. That said, holy shit it can do an absolute ton of busy work.
8 replies →
Because frameworks don’t have bugs? Or unpredictable dependency interactions?
This is generous, to the say the least.
2 replies →
> better to not have bugs in the first place
you must have never worked on any software project ever
1 reply →
In practice using someone else’s framework means you’re accepting the risk of the thousands of bugs in the framework that have no relevance to your business use case and will never be fixed.
1 reply →
> Unless you are only ever a single engineer, your career is filled with "I need to debug code I didn't write".
True, but there's usually at least one person who knows that particular part of the system that you need to touch, and if there isn't, you'll spend a lot of time fixing that bug and become that person.
The bet you're describing is that the AI will be the expert, and if it can be that, why couldn't it also be the expert at understanding the users' needs so that no one is needed anywhere in the loop?
What I don't understand about a vision where AI is able to replace humans at some (complicated) part of the entire industrial stack is why does it stop at a particular point? What makes us think that it can replace programmers and architects - jobs that require a rather sophisticated combination of inductive and deductive reasoning - but not the PMs, managers, and even the users?
Steve Yegge recently wrote about an exponential growth in AI capabilities. But every exponential growth has to plateau at some point, and the problem with exponential growth is that if your prediction about when that plateau happens is off by a little, the value at that point could be different from your prediction by a lot (in either direction). That means that it's very hard to predict where we'll "end up" (i.e. where the plateau will be). The prediction that AI will be able to automate nearly all of the technical aspects of programming yet little beyond them seems as unlikely to me as any arbitrary point. It's at least as likely that we'll end up well below or well above that point.
> Steve Yegge recently wrote about an exponential growth in AI capabilities
I'm not sure that the current growth rate is exponential, but the problem is that it doesn't need to be exponential. It should have been obvious the moment ChatGPT and stable diffusion-based systems were released that continued, linear progress of these models was going to cause massive disruption eventually (in a matter of years).
That doesn't change the fact that the submission is basically repeating the LISP curse. Best case scenario: you end up with a one-off framework and only you know how it works. The post you're replying to points out why this is a bad idea.
It doesn't matter if you don't use 90% of a framework as the submission bemoans. When everyone uses an identical API, but in different situations, you find lots of different problems that way. Your framework, and its users become a sort of BORG. When one of the framework users discovers a problem, it's fixed and propagated out before it can even be a problem for the rest of the BORG.
That's not true in your LISP curse, one off custom bespoke framework. You will repeat all the problems that all the other custom bespoke frameworks encountered. When they fixed their problem, they didn't fix it for you. You will find those problems over and over again. This is why free software dominates over proprietary software. The biggest problem in software is not writing the software, it's maintaining it. Free software shares the maintenance burden, so everyone can benefit. You bear the whole maintenance burden with your custom, one off vibe coded solutions.
I think back on the ten+ years I spent doing SRE consulting and the thing is, finding the problems and identifying solutions — the technical part of the work — was such a small part of the actual work. So often I would go to work with a client and discover that they often already knew the problem, they just didn’t believe it - my job was often about the psychology of the organization more than the technical knowledge. So you might say “Great, so the agent will automatically fix the problem that the organization previous misidentified.” That sounds great right up until it starts dreaming… it’s not to say there aren’t places for these agents, but I suspect ultimately it will be like any other technology we use where it becomes part of the toolkit, not the whole.
>I would argue that it's going to be the opposite. At re:Invent, one of the popular sessions was in creating a trio of SRE agents, one of which did nothing but read logs and report errors, one of which did analysis of the errors and triaged and proposed fixes, and one to do the work and submit PRs to your repo.
If you manage a code base this way at your company, sooner or later you will face a wall. What happens when the AI can't fix an important bug or is unable to add a very important feature? now you are stuck with a big fat dirty pile of code that no human can figure out because it wasn't coded by human and was never designed to be understood by a human in the first place.
I treat code quality, and readability, as one of the goals. The LLM can help with this and refactor code much quicker than a human. If I think the code is getting too complex I change over to architecture review and refactoring until I am happy with it.
What happens when humans can’t fix a bug or build an important feature? That is a pretty common scenario, that doesn’t result in the doomsday you imply.
1 reply →
Automatically solving software application bugs is one thing, recovering stateful business process disasters and data corruption is entirely another thing.
Customer A is in an totally unknown database state due to a vibe-coded bug. Great, the bug is fixed now, but you're still f-ed.
the other issue is "fixing" false-positives; i've seen it before with some ai tools: they convince you its a bug and it looks legit and passes the tests but later on something doesn't quite work right anymore and you have to triage and revert it... it can be real time sink.
„understand how these systems actually function" isn't incompatible with "I didn't write most of this code“
Except now you have code you didn‘t write and patches you din‘t write either. Your „colleague“ also has no long term memory.
Believe the re:Invent session is this one but correct me if I'm wrong: https://www.youtube.com/watch?v=rMPe622eGY0
> one to do the work and submit PRs to your repo
Have we not seen loads of examples of terrible AI generated RPs every week on this site?
Because nobody posts the good ones. They're boring, correct, you merge them and move on to the next one. It's like there's a murder in the news every day but generally we're still all fine.
Don't assume that when people make fun of some examples that there aren't thousands more that nobody cares to write about.
Was that session published online somewhere? I’d love to watch that.
An AI thing demoed well? Well, colour me shocked.
Amazon, which employs many thousands of SREs (or, well, pseudo-SREs; AIUI it's not quite the conventional SRE role), is presumably just doing so for charitable purposes, if they are so easy to replace with magic robots.
[dead]
The article gets at this briefly and moves on: "I can do all of this with the experience on my back of having laid the bricks, spread the mortar, cut and sewn for twenty years. If I don’t like something, I can go in, understand it and fix it as I please, instructing once and for all my setup to do what I want next time."
I think this dynamic applies to any use of AI, or indeed, any form of outsourcing. You can outsource a task effectively if you understand the complete task and its implementation very deeply. But if you don't, then you don't know if what you are getting back is correct, maintainable, scalable.
> instructing once and for all my setup to do what I want next time.
This works up to a point, but eventually your "setup" gets complicated, some of your demands conflict, or have different priorities, and you're relying on the AI to sort it out the way you expect.
But setups get equally complicated, even with human software engineers. The approach that the OP is talking about applies only to experienced, good architect-level SWEs, and I suspect that the code quality and its problems are going to be the same whether they are directing LLMs vs a set of junior SWEs to write the code.
There is an inherent level of complexity in projects that solve some real world problem, due to all the code handling edge cases that were added incrementally over time.
> any use of AI, or indeed, any form of outsourcing
Oh that's a good analogy/categorization, I hadn't thought about it in those terms yet. AI is just the next cheaper thing down from the current southeast asian sweatshop labor.
(And you generally get what you pay for.)
On the face of it, this or at least acting as a code reviewer from an experienced point of view seems like the solution, the problem is that we all naturally get lazy and complacent. I actually think AI was at its best for coding a year or so ago, when it could kind of do part of the work but theres no way you could ever ship it. Code that works today but breaks in 6 months is far more insidious.
It does beg, the question , whether any of this applies to less experienced people. I have a hunch that the open-ended nature of what can be achieved with AI will actually lead right back to needing frameworks, just as much as we do now, if not more, when it comes to less experienced people.
There are people out there who truly believe that they can outsource the building of highly complex systems by politely asking a machine, and ultimately will end up tasking the same machine to tell them how these systems should be built.
Now, if I were in business with any of these people, why would I be paying them hundreds of thousands, plus the hundreds of thousands in LLM subscriptions they need to barely function, when they cannot produce a single valuable thought?
I don't think there's going to be any catastrophic collapse but I predict de-slopping will grow to occupy more and more developer time.
Who knows, maybe soon enough we'll have specially trained de-slopper bots, too.
> Who knows, maybe soon enough we'll have specially trained de-slopper bots, too.
Fire, meet oil.
The naysayers said we’d never even get to this point. It’s far more plausible to me that AI will advance enough to de-slopify our code than it is to me that there will be some karmic reckoning in which the graybeards emerge on top again.
20 replies →
We already do. It's the same tool.
This sounds entirely too doomer.
There will obviously be companies that build a vibe coded app which too many people depend on. There will be some iteration (maybe feature addition, maybe bug fix) which will cause a catastrophic breakage and users will know.
But there will also be companies who add a better mix of incantations to the prompts, who use version control and CI, who ensure the code is matched with tests, who maintain the prompts and requirements documents.
The former will likely follow your projected path. The latter will do fine and may even thrive better than either traditional software houses of cheap vibe coding shops.
Then again, there are famous instances of companies who have tolerated terribly low investment in IT, including SouthWest Airlines.
I'd say you're absolutely right.
The problem is...what is the distribution of companies who do it "right" to companies that don't?
The same as the distribution of companies which are profitable over time and grow steadily, vs the others which clumsily flail around to somehow stay alive. To the winner go the spoils, and the winners will be a tiny fraction of companies, same as it ever was.
A way I look at it is that all net wealth creation in public companies has come from just 4% of businesses:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2900447
https://www.reddit.com/r/investing/comments/rk4udc/only_4_of...
It'll be similar with software companies. 4% of them will hit on a unique cultural and organizational track which will let them thrive, probably using AI in one form or another. The other 96% will be lucky to stay alive.
Same as it ever was.
I find that instructing AI to use frameworks yields better results and sets you up for a better outcome.
I use Claude Code with both Django and React which its surprisingly good with. I rather use software thats tried and tested. The only time I let it write its own is when I want ultra minimal CSS.
This. For area where you can use tested and tried libraries (or tools in general) LLMs will generate better code when they use them.
In fact, LLMs will be better than humans in learning new frameworks. It could end up being the opposite that frameworks and libraries become more important with LLMs.
> In fact, LLMs will be better than humans in learning new frameworks.
LLMs don't learn? The neural networks are trained just once before release and it's a -ing expensive process.
Have you tried using one on your existing code base, which is basically a framework for whatever business problem you're solving? Did it figure it out automagically?
They know react.js and nest.js and next.js and whatever.js because they had humans correct them and billions of lines of public code to train on.
4 replies →
> LLMs will be better than humans in learning new frameworks.
I don't see a base for that assumption. They're good at things like Django because there is a metric fuckton of existing open-source code out there that they can be trained on. They're already not great at less popular or even fringe frameworks and programming languages. What makes you think they'll be good at a new thing that there are almost no open resources for yet?
Yeah, I don't know why you'd drop using frameworks and libraries just because you're using an LLM. If you AREN'T using them you're just loading a bunch of solved problems into the LLMs context so it can re-invent the wheel. I really love the LLM because now I don't need to learn the new frameworks myself. LLMs really remove all the bullshit I don't want to think about.
LLMs famously aren’t that good at using new frameworks/languages. Sure they can get by with the right context, but most people are pointing them at standard frameworks in common languages to maximize the quality of their output.
4 replies →
How will LLM's become better than humans in learning new frameworks when automated/vibe coders never manually code how to use those new frameworks ?
1 reply →
The aspect of "potentially secure/stable code" is very interesting to me. There's an enormous amount of code that aren't secure or stable already (I'd argue virtually all of the code in existence).
This has already been a problem. There's no real ramifications for it. Even for something like Cloudflare stopping a significant amount of Internet traffic for any amount of time is not (as far as I know) investigated in an independent way. There's nobody that is potentially facing charges. However, with other civil engineering endeavors, there absolutely is. Regular checks, government agencies to audit systems, penalties for causing harm, etc. are expected in those areas.
LLM-generated code is the continuation of the bastardization of software "engineering." Now the situation is not only that nobody is accountable, but a black box cluster of computers is not even reasonably accountable. If someone makes a tragic mistake today, it can be understood who caused it. If "Cloudflare2" comes about which is all (or significantly) generated, whoever is in charge can just throw their hands up and say "hey, I don't know why it did this, and the people that made the system that made this mistake don't know why it did this." It has been and will continue to be very concerning.
Nobody is saying to skip testing the software. Testing is still important. What the code itself looks like, isn’t.
Knowing the inner working of a complex system is a hard requirement of its testing.
Testing is not a proof a software system is correct. Also, if tests are generated as well, there's no trust in how anything works or if the tests are covering important aspects.
Have you considered that betting against the models and ecosystem improving might be a bad bet, and you might be the one who is in for a rude awakening?
My favorite open source projects still have zillions of open bugs. Lots of projects are shutting down accepting external contributions because the PRs are so terrible. Wipro still isn't bankrupt.
Cheerleading is nice. Seeing the bug counts from my favorite projects significantly decrease would be evidence.
I'm not betting against them, I use them every day (but I don't "vibe code"—there's more intent). I'm just not treating them as a deity or other prayer-candle worthy entity. They're business tools. It's just a chat bot bro.
I agree. We've been assured by these skeptics that models are stochastic parrots, that progress in developing them was stalling, and that skills parity with senior developers was impossible - as well as having to listen to a type of self-indulgent daydreaming relish about the eventual catastrophes companies adopting them would face. And perhaps eventually these skeptics will turn out to be right. Who knows at this stage. But at this stage, what we're seeing is just the opposite: significant progress in model development last year, patterns for use being explored by almost every development team without widespread calamity and the first well-functioning automated workflows appearing for replacing entire teams. At this stage, I'd bet on the skeptics being the camp to eventually be forced to make the hard adjustments.
Pray tell, how has the world benefited from a flood of all these superhuman developers? Where is the groundbreaking software that is making our lives better?
1 reply →
I mean, if it ever gets good, eh, I suppose I'll use it? Pre-emptively using it in case it one day works properly seems rather perverse, tho.
I'm no fan of AI in terms of its long term consequences, but being able to "just do things" with the aid of AI tools, diving head first into the most difficult programming projects, is going to improve the human programming skills worldwide to levels never before imaginable
>is going to improve the human programming skills worldwide to levels never before imaginable
"We found that using AI assistance led to a statistically significant decrease in mastery. On a quiz that covered concepts they’d used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand"
https://www.anthropic.com/research/AI-assistance-coding-skil...
my comment is that it lowers the threshold to "just doing things"
An experiment where people "HAVE TO DO something" either way is testing something different
I know a fair amount about native android app development now because of using AI to build several native apps. I would know zero about about native android development if I had never attempted to build a native android app.
How would it improve skills?
Does driving a car improve your running speed?
I have to stretch your analogy in weird ways to make it function within this discussion:
Imagine two people who have only sat in a chair their whole lives. Then, you have one of them learn how to drive a car, whereas the other one never leaves the chair.
The one who learned how to drive a car would then find it easier to learn how to run, compared to the person who had to continue sitting in the chair the whole time.
1 reply →
I've found AI handy as a sort of tutor sometimes, like "I want to do X in Y programming language, what are some tools / libraries I could use for that?" And it will give multiple suggestions, often along with examples, that are pretty close to what I need.
No, but it does improve your ability to get to classes after work
An HN post earlier this week declared that “AI is killing B2B SaaS”:
https://news.ycombinator.com/item?id=46888441
Developers and businesses with that attitude could experience a similarly rude awakening.
If you've ever been responsible for large systems, you'll know why that is an incredibly naive position to take.
My expectation is that there'll never be a single bust-up moment, no line-in-the-sand beyond which we'll be able to say "it doesn't work anymore."
Instead agent written code will get more and more complex, requiring more and more tokens (& NPU/GPU/RAM) to create/review/debug/modify, and will rapidly pass beyond any hope of a human understanding even for relatively simple projects (e.g. such as a banking app on your phone).
I wonder, however, whether the complexity will grow slower or faster than Moore's law and our collective ability to feed the AIs.
Maybe software systems will become more like biological organisms. Huge complexity with parts bordering on chaos, but still working reasonably well most of the time, until entropy takes its course.
It's already like that, for a long time. Humans are quite capable of creating complex systems that become unwieldy the bigger they get. No one person can understand all of it. I will offer the AT&T billing system as an example that I'm all too familiar with as a customer, due to the pain it causes me. So many ridiculous problems with that system, it's been around a long time, and it is just so screwball.
4 replies →
Software engineers have been confidently wrong about a lot of things.
E.g. OOP and "patterns" in 90s. What was the last time you implemented a "visitor"?
P. Norvig mentioned most of the patterns are transparent in Common Lisp: e.g. you can just use a `lambda` instead of "visitor". But OOP people kept doing class diagrams for a simple map or fold-like operation.
AI producing a flawed code and "not understanding" are completely different issues. Yes, AI can make mistakes, we know. But are you certain your understanding is really superior?
Last time I used a visitor, it was probably last week when I created a lint rule. Visiting every node in a tree (ast or otherwise) with lambda is doing the pattern regardless of what you call the pattern. Tools like eslint still literally use the visitor pattern. I would point to software engineers dismissing tried and true ideas as the better generalization.
That means you don't understand visitor pattern: it's specifically an OOP pattern using OOP dispatch.
A tree traversal is a tree traversal.
Business has been operating on a management/executive culture for many decades now.
These people get paid millions a year to fly around and shake hands with people aka shit fuck all.
At times in the past I have worked on projects that were rushed out and didn't do a single thing that they were intended to do.
And you know what management's response was? They loved that shit. Ooooh it looks do good, that's so cool, well done. Management circle jerking each other, as if using everyone else's shafts as handles to climb the rungs of the ladder.
It's just...like it kills me that this thing I love, technology/engineering/programming...things that are responsible for many of the best things present in our modern lives, have both been twisted to create some of the worst things in our modern lives in the pursuit of profit. And the people in charge? They don't even care if it works or not, they just want that undeserved promotion for a job that a Simpsons-esque fucking drinking bird is capable of.
I just want to go back to the mid 2000s. ;~;
The hubris is with the devs that think like you actually.
Carefully, patiently, and intentionally in the direction of stability and good user experience?
The arrogance to think this isn't possible using llm tools and that the opposite didn't already exist at plenty of software shops pre-llm tools...
Every single CVE at least prior to the last couple years was written by a human. Azure and AWS have failed or have been compromised in embarrassing ways. I don’t take this human good, machine bad argument seriously. On the bright side, high integrity software may finally be an expectation for customers paying for software as “barely works” is becoming practically free.
I give a year, the realization would be brutal.
coding agents are an order of magnitude better than they were less than a year ago: Claude code, arguable the best, didn't exist a year ago and nonetheless, is an order better than when it did in late February.
It's really not a great sense of things for someone working in tech to make such a strong statement as the above given these circumstances. There is no reason to expect improvement will not continue, that agents' ability to review code-- drastically improved even since May/June, won't improve and, to the extend your concerns are valid about current agents, be able to manage the review and surfacing of issues for review and remediation.
This also ignores the growing awareness within the community of professional developers that, yes, blind trust-- as naively but minimally understandable given lack of knowledge on capabilities-- that blind trust cannot be given, and reasonable review of generated code at the end of a session is good practice. Developers are quite often already changing their habits with agents, if they were even avoiding such review before.
As for companies that may fire developers, rely near completely on agentic coding? I agree with the sentiment, but I really have not seen that happen to any great degree.
I won't assume this is the case for you, you may very well have dug in depth and worked with agents like claude code, understanding how best to scaffold by creating skills and agents, and still come to your conclusions. I would hope so. For the most part though, a large number of the people making comments of this sort seem to betray directly or indirectly that they-- nearly by definition of distrusting these agents and therefore avoiding them-- have little idea of what they are dismissing precisely because they have not taken the time to do more than dismiss them after little more than cursory review.
At the same time, many notable developers of significant accomplished near daily comment in pages here sharing experiences of true capability and assistance by these agents.
> the growing awareness within the community of professional developers that ... blind trust cannot be given, and reasonable review of generated code at the end of a session is good practice.
This saddens me. It saddens me that this is a trend among professional developers. It's an old lesson. We've known it for decades. Running arbitrary code is probably not the best idea.
You still "find most stuff by building/struggling". You just move up stack.
> there will be a shift when all of these vibe code only folks get cooked in a way that's closer to existential than benign
For those who are "vibe code only", perhaps. But it's no different than the "coding bootcamp only" developers who never really learned to think holistically. Or the folks who learned the bare minimum to get those sweet dotcom boom dollars back in the day, and then had to return to selling cars when it call came crashing down.
The winners have been, and will always be, those who can think bigger. The ones today who already know how to build from scratch but then find the superpower is in architecture, not syntax, and suddenly find themselves 10x more productive.
Come to the redteam / purpleteam side. We're having fun times right now. The definition of "every software has bugs" is now on a next level, because people don't even care about sql injection anymore. It's right built into every vibecoded codebase.
Authentication and authorization is as simple as POST /api/create/admin with zero checks. Pretty much every API ever slop coded looks like this. And if it doesn't, it will forget about security checks two prompts later and reverse the previously working checks.
Good lord. That's a level of carelessness that's just chef's kiss.
This article is a prime example. You‘re not foregoing frameworks. You‘re building your own. Except this time it‘s not „yours“, you let an AI build an unholy fusion of all frameworks in its dataset.
This comment ignores the key insight of the article. Design is what matters most now. Design is the difference between vibe coding and software engineering.
Given a good design, software engineers today are 100x more productive. What they produce is high quality due to the design. Production is fast and cheap due to the agents.
You are correct, there will be a reckoning for large scale systems which are vibe coded. They author is also correct, well designed systems no longer need frameworks or vendors, and they are unlikely to fail because they were well designed from the start.
>software engineers today are 100x more productive
Somebody needs to explain to my lying eyes where these 100xers are hiding. They seem to live in comments on the internet, but I'm not seeing the teams around me increase their output by two orders of magnitude.
I would say I'm like 1.2x more productive, and I think I'm more of the typical case (of course I read all of the code the LLM produces, so maybe that's where I've gone wrong).
If they did a year of work in ~3 days, presumably they're on a beach somewhere.
They are the people who have the design sense of someone like Rob Pike but lack his coding skill. These people are now 100x more capable than they were previously.
5 replies →
We’ve seen so many instances of Worse Is Better™ materialize around software development (and other areas) that I don’t dare to be as optimistic as you.
The interesting aspect is: that allows people who couldn't do it before to build tools for their department needs. Where in the past they would have needed a budget for going to the IT department and then do complex processes to transfer the requirements from business needs into a form the developers understand.
Now they can quickly build simple tools. Yes, they are bad from software quality pov, but they solve problems.
> "Now they can quickly build simple tools. Yes, they are bad from software quality pov, but they solve problems."
Simple tools can make big mistakes.
I've done it. It was an outlier case. A small mistake of logic giving out money already paid once in a neglected corner (so to speak) of the logic tree. One bad script can overshadow tens of good ones.
I think it would be the opposite and we are all in for a rude awakening. If you have tried playing with Opus 4.6 you would know what I am talking about.
I tell my colleagues we're in the instantaneous peak of the AI developer relationship, especially for code monkeys. We're still valued, still paid really well, and our jobs will get easier and easier probably for the next 5-10 years! After that, maybe not so great for many of us, with the developers that use software as a means of their actual profession continuing to do just fine (hard math/science/optimization/business planning/project planners/etc).
I think it's going to be an amazing shift from those that know intricate details of software to enabling those that have the best ideas that can be implemented with software (a shift from tool makers to tool users).
I think many developers misunderstand the quality of software that people outside of software are willing to live with, if it does exactly what they want when they need it. For a user, it's all black box "do something I want or not" regardless of what's under the hood. Mostly "academic", things like "elegant" and "clean" and "maintainable" almost never practically matter for most practical solutions to actual problems. This is something I learned far too late in my professional career, where the lazy dev with shite code would get the same recognition as the guy that had beautiful code: does it solve the real world problem or not?
Safety critical, secure, etc, sure, but most is not. And, even with those, the libraries/APIs/etc are separate components.
But by then many of us are already starved. That’s why I always said that engineers should NOT integrate AI with internal data.
What's makes you so sure of your statement?
I have be building systems for 20 years and I think the author is right.
I've been building systems for 40 years, and I think... you know what, nevermind, it doesn't really matter what either of us thinks.
Observation.
> A significant number of developers and businesses are going to have an absolutely brutal rude awakening in the not too distant future.
I pray (?) for times like the ones you predict. But companies can stay irrational longer than the average employee can afford.
>A significant number of developers and businesses are going to have an absolutely brutal rude awakening in the not too distant future.
Correct. Those who wave away coding agents and refuse to engrain them into their workflows are going to be left behind in the dust.
> Correct. Those who wave away AI and refuse to engrain it into their workflows are going to be left behind in the dust.
Similar to those who waved away crypto and are now left behind in the dust, yes?
Might not be the best counter example since everyone who has bought BTC before Jan 2024 is now in massive profit.
2 replies →
You forgot NFTs
1 reply →
I think Bitcoin and major cryptos outperformed a lot of assets over the last decade, so you could say it left some people behind in the dust, yes
3 replies →
[dead]
Doubt on that. AI usually only wastes time and produces bugs.
> bbut you're holding it wrong, just two more prompts and three more agents and it will be a real boy
So, you invented an IDE, except more opaque and expensive? Welcome to the club.
You both are likely incorrect, the answer lies in the middle rather than the extremes
3 replies →
Yeah I completely disagree with the author actually, but also with you.
The frameworks are what make the AI write easily understandable code. I let it run nextjs with an ORM, and it almost always creates very well defined api routes, classes & data models. etter then I would do often,
I also ask it be way more correct on the validation & error handling then I would ever do. It makes mistakes, I shout at it and corrects quickly.
So the project I've been "vibe coding" have a much better codebase then I used to have on my solo projects.
Why does there seem to be such a divide in opinions on AI in coding? Meanwhile those who "get it" have been improving their productivity for literally years now.
I think there are a number of elements:
- What you are working on. AI is better at solving already solved problems with lots of examples.
- How fast/skilled you were before. If you were slow before then you got a bigger speed up. If AI can solve problems you can’t you unlock new abilities
- How much quality is prioritized. You can write quality, bug free code with AI but it takes longer and you get less of a boost.
- How much time you spend coding. If a lot of your job is design/architecture/planning/research then speeding up code generation matters less
- How much you like coding. If you like coding then using AI is less fun. If you didn’t like coding then you get to skip a chore
- How much you care about deeply understanding systems
- How much you care about externalities: power usage, data theft, job loss, etc.
- How much boilerplate you were writing before
I’m sure that’s not a complete list but they are a few things I’ve seen as dividers
> How much you like coding. If you like coding then using AI is less fun.
I'm surprised this is never brought up here on "Hacker" News. I've been reading HN for 14 years and all this time I thought people here enjoyed programming. Turns out the majority hates it apparently.
1 reply →
A few more:
- How much do you prioritize speed?
- Do you have a big backlog of dev tasks ready to go?
- What are the risks if your software doesn’t work?
- Are you working on a green field or legacy project? Prototypes or MVPs?
1 reply →
I've been using it every day for nearly two years now, with your suggested productivity boost observed.
The difference is that I'm not just letting agents willy nilly commit code. I treat them more like a companion and guide their steps (I use Cline w/ Sonnet/Opus 4.5/4.6). Not only do I save a ton of money on tokens, but the results end up being infinitely better than the "yolo" mode outcomes (even with excellent prompting/context).
From my POV, the only divide is between a willingness to be an accountable professional versus someone who just "lets the AI do it" and whistles-with-hands-in-pockets when that code inevitably blows up in a way you couldn't predict (because you weren't checking, only the AI was, which you swore was "good enough").
That approach works if you're just sitting on your couch hacking up toys to dice roll on X. But if you're trying to build reliable, deterministic systems that aren't constantly buzzing you awake at 3am, you're asking for a serious humbling if no one in your organization can explain how or why anything your business relies on works the way it does (that's operationally suicidal, imo, but hey—America).
That gets misinterpreted as being a "luddite," when really it's just having been down the rabbit hole enough times to know that if you can't point to and understand why it's happening (and ideally, whodunit), you don't know shit.
There's a lot of reasons. There's a lot of breadth to "software engineering" (FAANG, web dev, embedded, OS, small business, etc.)
I'm sure there are some places where LLMs are bad due to lack of training data. There are some places where the LLMs are bad because the code base is terrible (and there's always "rockstars" at these jobs that severely overestimate their skills because they're always the one fixing the mess... which they also probably caused). Some devs/engineers feel threatened. Many devs/engineers think they're special and super smart so surely no machine can do their job.
The hard thing to watch is developers grapple with the idea that what makes them special isn’t so special anymore.
Clearing out the digital asbestos is going to make a lot of consultants a lot of money.
Possible, but this sounds like motivated reasoning.
The future is already here. Been working a few years at a subsidiary of a large corporation where the entire hierarchy of companies is pushing AI hard, at different levels of complexity, from office work up through software development. Regular company meetings across companies and divisions to discuss methods and progress. Overall not a bad strategy and it's paying dividends.
A experiment was tried on a large and very intractable code-base of C++, Visual Basic, classic .asp, and SQL Server, with three different reporting systems attached to it. The reporting systems were crazy being controlled by giant XML files with complex namespaces and no-nos like the order of the nodes mattering. It had been maintained by offshore developers for maybe 10 years or more. The application was originally created over 25 years ago. They wanted to replace it with modern technology, but they estimated it'd take 7 years(!). So they just threw a team at it and said, "Just use prompts to AI and hand code minimally and see how far you get."
And they did wonderfully (and this is before the latest Claude improvements and agents) and they managed to create a minimal replacement in just two months (two or maybe three developers full time I think was the level of effort). This was touted at a meeting and given the approval for further development. At the meeting I specifically asked, "You only maintain this with prompts?" "Yes," they said, "we just iterate through repeated prompts to refine the code."
It has all mostly been abandoned a few months later. Parts of it are being reused, attempting a kind of "work in from the edges" approach to replacing parts of the system, but mostly it's dead.
We are yet to have a postmortem on this whole thing, but I've talked to the developers, and they essentially made a different intractable problem of repeated prompting breaking existing features when attempting to apply fixes or add features. And breaking in really subtle and hard to discern ways. The AI created unit tests didn't often find these bugs, either. They really tried a lot of angles trying to sort it out - complex .md files, breaking up the monolith to make the AI have less context to track, gross simplification of existing features, and so on. These are smarty-pants developers, too, people who know their stuff, got better than BS's, and they themselves were at first surprised at their success, then not so surprised later at the eventual result.
There was also a cost angle that became intractable. Coding like that was expensive. There was a lot of hand-wringing from managers over how much it was costing in "tokens" and whatever else. I pointed out if it's less cost than 7 years of development you're ahead of the game, which they pointed out it would be a cost spread over 7 years, not in 1 year. I'm not an accountant, but apparently that makes a difference.
I don't necessarily consider it a failed experiment, because we all learned a lot about how to better do our software development with AI. They swung for the fences but just got a double.
Of course this will all get better, but I wonder if it'll ever get there like we envision, with the Star Trek, "Computer, made me a sandwich," method of software development. The takeaway from all this is you still have to "know your code" for things that are non-trivial, and really, you can go a few steps above non-trivial. You can go a long way not looking to close at the LLM output, but there is a point at which it starts to be friction.
As a side note, not really related to the OP, but the UI cooked up by the LLMs was an interesting "card" looking kind of thing, actually pretty nice to look at and use. Then, when searching for a wiki for the Ball x Pit game, I noticed that some of the wikis very closely resembled the UI for the application. Now I see variations of it all over the internet. I wonder if the LLMs "converge" on a particular UI if not given specific instructions?
These are the blog posts we need.
This is the siren song of llm. "Look how much progress we made"
Effort increases as time to completion decreases. The last 10% of the project takes 90% of the effort as you try to finish up, deploy,integrate and find the gaps.
Llms are woefully incapable of that as that knowledge doesn't exist in a markdown file. It's in people's heads and you have to pry it out with a crowbar or as happens to so many projects, they get released and no one uses it.
See Google et Al. "We failed to find market fit on the 15th iteration of our chat app, we'll do better next time"
For complex code bases generated by AI the last 10% takes 190% of the effort because you end up tearing it all apart to rebuild it right.
I've noticed this in my small scale tests. Basically the larger the prompt gets (and it includes all the previously generated code because that's what you want to add features to), the more likely is that the LLM will go off the rails. Or forget the beginning of the context. Or go into a loop.
Now if you're using a lot of separate prompts where you draw from whatever the network was trained on and not from code that's in the prompt, you can get usable stuff out of it. But that won't build you the whole application.
> I wonder if the LLMs "converge" on a particular UI if not given specific instructions?
Purple. They really fucking like this purple gradient background for some reason lol.
In a veritable ocean of opinions it is excellent to see a detailed, first-hand report. Many thanks!
Most systems are not needed anyway. They are built to be built, by companies not caring for how good, secure, or fast they are, or even how usable they are, as long as users are lured or forced to use them.
Nothing speciail will happen when things are built "this way". People will just be used to lower quality, like they've been used to enshittified software for the past two decades.
No "artisans" will be employed to fix any mess. The mess will be our software baseline going forward.
Nah, do you honestly think the thousands of sub-packages in your project are actually being maintained? False sense of security there.
> Good time to be in business if you can see through the bs and understand how these systems actually function
You missed out the most crucial and least likely requirement (assuming you're not self employed); management also need to be able to see through the bs.
[dead]
Back in the 00s people like you were saying "no one will put their private data in the cloud!"
"I am sick of articles about the cloud!"
"Anyone know of message boards where discussing cloud compute is banned?"
"Businesses will not trust the cloud!"
Aside from logistics of food and medicine, most economic activity is ephemeral wank.
It's memes. It's a myth. Allegory.
These systems are electrical state in machines and they can be optimized at the hardware layer.
Your Python or Ruby or whatever you ship 9,000 layers of state and abstraction above the OS running in the data center has little influence on how these systems actually function.
To borrow from poker; software engineers were being handed their hat years ago. It's already too late.