Comment by Waterluvian
6 hours ago
I had an Iron Man moment last week where I was “vibe coding” a UI design with component tests live on the other screen. Iterating by asking it to move things, reduce emphasis of an element, exploring layout options, etc. The loop was near realtime and felt amazing.
The code it generated was awful. The kind of garbage that people who don’t know any better would ship: it looked right and it worked. But it was instantly a maintenance dead end. But I had an effortless time converging on a design that I wouldn’t have been able to do on my own (I’m not a designer). And then I had a reference design and I manually implemented it with better code (the part I am good at).
This isn't an indictment of how good AI is but how poor our tools are. We had gui makers in 1996 that made slop which allowed you to iterate in real time. They didn't need a datacenter worth of compute and a nuclear reactors worth of power to run.
Could it be that the fact that the thing you’re an expert at looked like garbage to you, but the things you’re not an expert at, looked just fine, is not a coincidence?
You can talk to a bunch of designers who will say the opposite. Claude Design Studio generated this garbage UI, that I fixed manually, but it created great code j never could have that made it work.
This is the juxtaposition the general public is in. They don’t have advanced tech skills to know any better so they see an output that they can’t produce from their skills and think it’s great. Maybe it is, maybe it isn’t. What does the code look like?
Both had a working prototype. The flaw everyone is making is that they are over focusing on the artifact and not that they have a shared tangible object that they can both editorialize and iterate on.
These systems should allow rapid iteration on discovery and thinking. One can now make a prototype a day that would have taken a week. That means that we should be able to converge on a much better design in the same amount of time it would have taken to make a v0 that turns how to have systemic flaws.
AI should scale our understanding of systems, not just shovel out half baked features and apps.
1 reply →
Maybe specialists have a higher bar than consumers, and as a design consumer he's right about the design, and the designer is right about the code, if "being right" means "understanding what the end customer will think about this".
I think this is true, it's like a close relation of the Gell-Mann amnesia effect
https://en.wikipedia.org/wiki/Michael_Crichton#:~:text=%5B14...
More robust link (to the heading by ID, rather than by text directive with pre/post text that will change): https://en.wikipedia.org/wiki/Michael_Crichton#%22Gell-Mann_...
This!
People are never perfectly even in intelligence across all possible disciplines.
It's worth pointing out that Crichton coined that term during a period in his life where he was rapidly descending into conspiracy and iconoclastic thought, and this is of a piece with that.
Gell-Mann's observation was a sincere and thoughtful caution about the way we transmit information about complicated ideas. Crichton's "amnesia effect" is an excuse to ignore media you dislike.
I'm confused, this doesn't make sense. The target they're iterating on (UI) is the same one whose quality they're assessing, not a different one (source code).
You're suggesting that (a) their UI skills are lacking (based on what? isn't UI exactly what they were iterating on and trying to improve?), and (b) that a real UI expert would've somehow felt the UI they were working on was consistently garbage, despite how many times they iterate on it?
Which means you're saying you don't believe anyone can actually produce high quality (to an expert) output with AI on the same target they're working on, and if they think they are, that just means they don't have a good sense of quality?
It's not confusing. It makes sense.
1 reply →
Without proper training, what looks good may be trash. I always thought pixel art generated by diffusion models looked damn good. Then I started watching and reading reviews by actual pixel artists, and all they saw was flaws. And it wasn't just nitpicking, it was things that were fundamentally wrong, difficult to fix and would look awful and amateurish and distracting to the player in production.
1 reply →
this is why people still enjoy eating at Olive Garden and Chipotle and Sweetgreen
basically the AI-slop version of food, yet still they thrive
Ai is a hammer. Use it right and it makes you very powerful. But it's not an easy tool.
Good point on "Gell-Mann Amnesia Effect."
> The code it generated was awful. The kind of garbage that people who don’t know any better would ship: it looked right and it worked. But it was instantly a maintenance dead end.
In the Tailwind thread the other day I was explicitly told that the intended experience of many frameworks is "write-only code" so maybe this is just the way of the future that we have to learn to embrace. Don't worry how it's all hooked up, if it works it works and if it stops working tell the AI to fix it.
It's kind of liberating I guess. I'm not sure if I've reached AI nirvana on accepting this yet, but I do think that moment is close.
I'm pretty you wouldn't want the same for code that runs healthcare, banks or transport. Only useless shitty web projects could embrace what's you're saying. And no there's no "Claude review the code and improve it" magical formula
I work in the health software space and there are tons of internal tools which aren't production code that can benefit massively from throwaway "write-only code". Putting a web UI on top of a management CLI tool so support ops can run things without needing an oncall engineer can be a huge win. I recently built a testing UI that doubles a demo-scenario-setup tool. Is it well-engineered? Who cares - it pokes the right things into the database and runs the right backend tasks, and has helped me catch and fix dozens of real bugs in the UIs that customers see.
There is an enormous untapped market for crappy low-effort apps which previously weren't worth the time - but with the effort so low put together a simple dashboard or one-off tool it becomes much more attractive.
The problem is it’s impossibly hard to test all the edge cases
Which is probably why so many random buttons in microsoft/apple/spotify just stop working once you get off the beaten path or load the app in some state which is slightly off base
The problem is worse than that.
The number of edge cases in a software is not fixed at all. One of the largest markers of competence in software development is being able to keep them at minimum, and LLMs tend to make that number higher than humanely possible.
Yeah, the biggest thing I've noticed from LLMs is that large tech products now have even more bugs. Turns out the humans weren't so bad after all...
4 replies →
Easy, have Claude review the code, tell it to be critical and that it needs to be easier to understand, follow Clean Code, SOLID principles and best practices. Lie to it, say you got this from a Junior developer, or "review it as if you were a Staff Level Engineer reviewing Junior code" the models can write better code, just nobody tells them to.
Lol, the only thing worse than a junior developer following Clean Code and SOLID has to be an LLM messing with code so it looks like it follows.
1 reply →
Code review is the main thing I use LLMs for. I have found it to be remarkably candid when you tell it the code came from another LLM (even name it). I was running Kimi K2.6 Q4 locally, seeing if it could SIMD a bit-matrix transpose function, and it was slow enough that I would paste its thinking into Gemini every few minutes. Gemini was savage.
1 reply →
This is it. I've had a similar experience in just playing around I asked it to clean up some code it wrote to increase maintainability and readability by humans. After a few iterations it had generated quite solid code. It also broke the code a couple of times along the way. But it does get me thinking that these pipelines with agents doing specific tasks makes a lot of sense. One to design and architect, one to implement, one to clean, one to review, one to test (actually there's probably a bunch of different agents for testing -- testing perf/power, that it matches the requirements/spec, matches the design, is readable/maintainable, etc...).
1 reply →
Why wouldn't Claude just impose this same loop in the code it writes - or better, write better code before it needs such review?
3 replies →
Even better, if you have access to multiple models, tell it you got the code from another AI agent.
I did an experiment on this a few weekends ago and Codex for example was a lot more adversarial and thorough in its review when given Claude-authored code compared to when given the same code with "I wrote this, can you review it?"
1 reply →
Is this a joke? Smartest people on the planet never thought about telling AI to just write better code?
5 replies →
I have been wondering recently that if the cost of just throwing everything out and building it from scratch again gets low enough, maybe maintainability becomes less of a priority? Can we just embrace the thing like those Zen carpenters who build wooden fire shrines do where they just accept that the thing will keep burning down and they make a discipline around getting really good at rebuilding it?
Granted, the load bearing thing here is whether we’re actually getting good at rebuilding up to any sort of standard of quality. Or if the tooling is even structurally capable of doing that rather than just introducing new baskets of problems with each build.
I'm looking at that Tailwind thread. Do you really think that your comment here is a fair assessment of what you were told there? Come on now.
https://news.ycombinator.com/item?id=48166334
We collectively have to re-learn what operations are expensive and what are cheap.
Prototypes are practically free now. You can ask the AI try each architectural or stylistic option and just see which code you like better.
To your point, another interesting note is that rewriting and rearchitecting are also very good.
One pattern I like is to vibe code a set of solutions, pick the approach, then backfill tests and do major refactors to make it maintainable.
Here the skill is knowing what good architecture looks like, and knowing how to prompt and validate (eg what level of tests will speed up the feedback cycle or enable me to make the LLM’s changes legible).
To be fair the “ready, fire, aim” approach of rapid prototyping has been known for a long time, but you need to be quite quick at coding in old world for it to work well IMO.
Free? Lol
I wonder how much of this is momentum.
At the moment, we understand the basic tech, could reasonably DIY, but choose not to knowing full well there's a mess of understandable code somewhere we could go clean up but dont want to. We accept fast iterations because we know roughly the shape of how it "should be" and can guide an automated framework towards that. This is especially true on our own projects or something we built originally! Stark/Iron man knew/moved, the suit assisted by adding momentum.
We're riding our "knowledge momentum".
If companies can hold out long enough, that knowledge completely fades, and the tool is all you have. At that point, they are locked in. Then it's not Iron man, it's an Iron lung (couldn't resist!)
The blog post is spot on: AI is exactly like an Iron Man suit - if you never take it off your muscles will atrophy to nothing!
Yeah that’s my main concern. It feels so so easy to be lazy and do a bad job now. And then my skills weaken and what makes me valuable fades.
I love the Iron lung reference. Perfect.
That's the model I've arrived to as well:
- first I've created a skill how the architecture of the system should look like
- I'll tell the LLM to follow the guidelines; it will not do that 100%, but it will be good enough
- I'll go through what it produced, align to the template; if I like something (either I've not thought about the problem in that way, or simply forgot) I add that to the skill template
- rinse and repeat
This is not only for architecture of the system, but also when (and how to) write backend, frontend, e2e tests, docs. I know what I want to achieve = I know how the code should be organized and how it should work, I know how tests should be written. LLMs allow me to eliminate the tediousness of following the same template every time. Without these guardrails it switches patterns so often, creating unmaintainable crap
Bear in mind - the output requires constant supervision = LLM will touch something I told it not to touch, or not follow what I told it to do. The amount of the output can also sometimes be overwhelming (so, peer review is still needed), but at this point I can iterate over what LLM produces with it, with another LLM, then give to a human if it together makes sense
> But I had an effortless time converging on a design that I wouldn’t have been able to do on my own (I’m not a designer).
I'm not a designer either, but I've been around designers long enough to recognize when something is bad but just not know what is needed to make it better/good. I've taken time to find sites that are designed well and then recreated them by hand coding the html/css to the point that I consider myself pretty decent at css now. I don't need libraries or frameworks. My css/html is so much lighter than what's found in those frameworks as well. I still would not call myself a designer, but pages look like they were designed by a mediocre designer rather than an engineer :shrug:
I'm trying to test if vibe coding can actually scale... And man is it painful.
AI is great at creating slop that almost works.
But, my god, it is terrible at following clear as day instructions on how to cleanup slop.
It wrote 150k lines of code that almost works in 2 months. It's taken 1 month to delete about 2000 lines of broken architecture and fix it, and it still hasn't gotten it done, despite nonstop repeated efforts to do something not that hard.
I definitely could've fixed it less time then I've spent prompting at this point (but no way I'd have gotten the other 150k lines). But doing it myself is not the point. It's to see if it can actually scale.
The answer is yes... But my god is it agonizing.
The creating garbage part that almost works is fun.
The inevitable cleanup is not.
And unfortunately I don't see this aspect materially improving in the short term.
If you want it to code you something about 5-10k lines of code that's already been done 1000 times before or only slightly different, it's great.
Most people want more than that.
Why was it a maintenance dead end? It sounds like you were able to iteratively work on it in its current state, but are you going to be the one maintaining the code?
I keep asking myself the same questions, and the conclusion I keep coming to is the clean modeled structure we want to see is for humans to maintain and extend, but the AI doesn't need this.
There's definitely an efficiency angle here where it's faster for AI to go from a clean modeled solution to the desired solution because it's likely been trained on cleaner code. Is this really going to matter though?
The best argument I can come up with is the clean modeled solution is better for existing development tools because it's less likely to get confused by the patch work of vibes throughout the code; but this feels like it ultimately becomes an efficiency concern as well.
This just might be the new reality, and we need to stop looking behind the curtain and accept what the wizard presents us.
> the clean modeled structure we want to see is for humans to maintain and extend, but the AI doesn't need this.
This does not match my experience. I do a lot of AI-assisted coding at this point, and what I've seen is that when the AI is asked to extend or modify existing code, it does a much better job on clean, well-structured and well-abstracted code.
I think the reason is simple, and tracks for humans as well: well-structured code is simply easier to understand and reason about, and takes a smaller amount of working-set memory. Even as LLMs get better with coding, I expect that they would converge on the same conclusion, namely that good structure + good abstractions make for code that is more efficient to work with.
Yeah I have had claude take over multiple internal (human written) projects that were in a dire state and spent a week just completely refactoring them and adding exhaustive tests before doing any new features. It's worth starting from a clean slate.
I keep hearing the assertion that you can’t make high quality, maintainable code with LLMs. The last two years using AI have shown me exactly the opposite.
I think it’s all about the structure you use to work in and how you use the model. We are shipping better, more human friendly code, with less bugs, then we ever did before and doing it at 1/10 the cost before LLMs.
But we are definitely not vibe coding, and the key seems to be devs with years of experience managing teams, managing the LLM instead. Basically you create the same kind of formal specifications, conventions, and documentation that you would develop for a project with two or three teams, then use that to keep the project on the rails recursively looping back through the docs as you go along. I’ve only had to back out of a couple of issues over the last year, and even though that cost a couple of hours, it was still extremely cheap.
Meanwhile we are shipping at 4x speed with 1/4 the labor, and the code is better than it was because the “overhead” of writing maintainable, self documented code has inverted into the secret ingredient to shipping bug free code at unprecedented speed.
If you just explain the standards to which you want the code written, use a strict style guide, have a separate process that ensures test coverage (not in the same context) you can get example quality code all the way through. Turns out that’s also in the training data.
Many of us recognize that the days of nearly-free tokens is quickly drawing to a close, and at some point humans may very well have to dig their keyboards out of cold storage and return once again to the code mines.
OAI and Anthropic need to generate cash flows from operations - once they go public that’s it. Any future funding for reinvestment has to come from internal funds beyond existing raising + IPO.
So yeah, it’s imminent. Let’s see how demand shifts in response in the future.
1 reply →
The reason why you will never get software engineers (in companies) to accept the man behind the curtain is liability. If a human software engineer is still responsible for what happens when the AI developed code has a catastrophic bug or security vulnerability, then the only way for the human to know if there is a problem is to be able to read through the code or run it through some <insert advanced formal verification tool here> that guarantees zero issues.
I think we eventually end up at the tool approach via vendors providing the tools to other companies, but it still feels like there's a long road ahead to get there.
> but the AI doesn't need this
That's not true. The LLM performance will degrade as the codebase gets messier as well. You get to a point where every fix breaks something else and you can't really make forward progress.
Yes, you might be able to get a bit further with a messy codebase just because the LLM won't complain and will just grind through fixing things, but eventually it will just start disabling failing tests instead of actually fixing things.
The token cost to fix might surpass what a human would cost to just do it.
Sometimes I think the main value in AI-maintained code being “high quality” is when the structure can enforce invariants. If invalid states aren’t representable, then the AI can’t easily add bugs in the future.
Of course that just leads to: what’s the best way to achieve that goal? Through elegant code or adding lots of tests? Which is a debate from long before LLMs existed.
>This just might be the new reality, and we need to stop looking behind the curtain and accept what the wizard presents us.
This is how societies become shittier. People who are ostensibly responsible for doing their jobs not giving a damn about quality.
> Why was it a maintenance dead end?
LLMs have a limit to how deep they can understand and refactor architectural issues.
That limit is far, far lower than a human's.
Tangent: I never learned how to make the sorts of websites people find "professional" or "pretty" I could make functional and easy-to-use webapps, but not something people would think looks good or like something they would want to use. LLMs crushed this, without performance overhead; can still be HTML/CSS/targetted JS.
I feel the same but the question I struggle most with is this: "Does it matter when the people who are going to come along and maintain this are just going to use AI to fix or adjust this maintenance nightmare?"
At that point the code becomes a compile target, and then you need a new source of truth.
Which I think is perfectly worthy of exploration. Some people want to check in the prompts. Or even better, check in a plan.md or evenest betterest: some set of very well-defined specifications.
I'm not sure what the answer will be. Probably some mix of things. But today it is absolutely imperative that the code I write for the case I wrote it in is good quality and can be maintained by more than just me.
When we want to maintain a reliable, stable "product" in traditional software development (a binary executable artifact that ships out to users, or the binary engine of some SaaS the company sells to users), we don't just check in (to the source of truth repo) the actual application-layer source code. We also check in build instructions (think autoconf/cmake/etc) and have some concept of compiler compatibilities / versions, build environments, and papering over their runtime differences. And then our official executable output is not just defined by "Tag v1.23.45 of the application source code repo" - it's additionally defined by the build environment (including, critically, the compiler version, among many others).
It's tempting to move out a layer and try making prompts and plan.md the "source code", and then the generated actual-source-code becomes just another ephemeral form of "intermediate representation" in the toolchain while building the final executable product. But then how are you versioning the toolchain and maintaining any reasonable sense of "stability" (in terms of features/bugs/etc) in the final output?
Example: last week, someone ran our "LLM inputs" source code through AgentCo SuperModel-7-39b, and produced a product output that users loved and it seemed to work well. Next week, management asks for a new feature. The "developer" adds the new feature to the prompting with a few trial iterations, but the resulting new product now has 339 new subtle bugs in areas that were working fine in last week's build owing the fact that, in the meantime, AgentCo has tweaked some weights in SuperModel-7-39b under the hood because of some concern about CSAM results or whatever and this had subtle unrelated effects. Or better yet: next month, management has learned that OtherCo MegaModel-42.7c seems to be the new hotness and tells everyone to switch models. Re-building from our "source" with the new model fixes 72 known bugs filed by users, fixes another 337 bugs nobody had even noticed yet, and causes 111 new bugs to be created that are yet-unknown.
If you treat the output source code as a write-only messy artifact, and you don't have stable, repeatable models, and don't treat model updates/changes as carefully as switching compiler vendors and build environments, this kind of methodology can only lead to chaos.
And don't even get me started on the parallel excuses of "Your specifications should be more-perfect" (perfection is impossible), or "An expansive testsuite should catch and correct all new bugs" (also impossible. testing is only as good as the imperfect specification, and then layers in its own finite capabilities to boot).
I don't see the benefit of checking in either prompts or specs.
I never tried spec driven development for myself, but if I review other's MRs I am typically exhausted after the first 10 lines.
And there are hundreds of lines, nearly always with major inaccuracies.
For myself I always found the plan mode to work well. Once the implementation is done, the code is the source of truth. If it works, it works.
When I want to add more functionality or change it, I just tell the agent what I want changed.
I doubt walls of semi-accurate existing specs are going to be beneficial there, but maybe my work differs from yours.
3 replies →
Same. Messy code makes it harder for us to understand and thus maintain the code (which is why people often refer to code as a liability), but is that the case for AI tools as well? If not, it seems like clean code may not matter as much anymore.
The problem with crappy frontend code is not only the maintenance. It's that stuff such as responsive design, accessibility or cross-browser compatibility that work nearly for free with elegant code won't work at all.
The problem is that technical debt is compounding. Bad LLM architectural and implementation decisions just blend in to the background and you build layer upon layer of a mess. At some point it becomes difficult and expensive (token wise) to maintain this code, even for an agent.
I mitigate this by few things: 1. Checkpoints every few days to thoroughly review and flag issues. Asking the LLM to impersonate (Linus Torvalds is my favorite) yields different results. 2. Frequent refactors. LLMs don't get discouraged from throwing things out like humans do. So I ask for a refactor when enough stuff accumulates. 3. Use verbose, typed languages. C# on the backend, TypeScript on the frontend.
Does it produce quality code? Locally yes, architecturally I don't know - it works so far, I guess. Anyway, my alternative is not to make this software I'm writing better but not making it at all for the lack of time, so even if it's subpar it still brings business value.
> The code it generated was awful.
I suppose you could solve that in two ways. Manually rewrite it as you did. Or formalize an architecture and let the AI rewrite it with that in mind. I suspect that either works.
I suspect at some point AI-written code will be eventually artifacts generated build-to-build. The design docs and UI tests are the source and the model follows instructions to generate the product. If you make the models deterministic then model improvements give you code improvements across your entire codebase "for free".
Very recognizable and hard to reason about! I did something similar, but while looking at the code, it looked so procedural, hardly abstract, no vision. How was I ever going to maintain this? I guess Ai will do it forever?
Did you try to have it clean up the code and refactor? I find while the code is usually low to mid tier that it’s a lot better than the first pass. I of course back up the working version lol. Usually I can coax something better out of it
If you are just blindly vibe coding without any parameters, guardrails, architecture, or broad guidance; you're going to have a mess of slop.
the power comes from creating the machine you can steer. Treat AI like an over eager college intern who you need to hand hold, but do tasks.
I used AI to write code (features) then I used AI to refactor the architecture using best practices and get rid of the technical debt. I don't remember the last time I modified code by hand honestly.
This is the part people are missing. I spend 30-40% of my time "vibe coding" doing "vibe clean up". It's fine, I'm still 10x more productive than I ever was.
> But it was instantly a maintenance dead end.
I gave up on this recently. It achieved the goal now, and in a year or two, when you actually want to add whatever feature, the SOTA AI will probably be able to clean it up as it does so. What does "maintain" even mean anymore?
If you don't agree, how many years into the future do we need until you would agree?
The problem is that people keep saying this, but the code keeps being bad. Every time I commit myself to trying to build something with AI, I end up wasting a ton of time and backing it out or completely rewriting it without the AI. The code it generates just isn't where it needs to be.
And people have been saying this exact thing for years now. Someone said this very thing two years ago. And we're still at the "maintenance dead end" stage. So let me flip it back on you: how many years are we going to pour an obscene amount of resources into this thing that is always going to be able to clean up its own messes "in a year or two" before we realize its a dead end (at best) and we need to be using those resources elsewhere? And, similarly, what happens to you when the SOTA AI in two years can't clean up the code it wrote for you two years ago, but people are depending on it and your still on the hook for maintaining it?
> If you don't agree, how many years into the future do we need until you would agree?
Respectfully, I asked first. ;)
> before we realize its a dead end (at best)
You've declared the future, which doesn't leave much room for a conversation. So, cheers!
> The kind of garbage that people who don’t know any better would ship: it looked right and it worked.
I feel what you write, but then again: every now and then i write small greasmonkey scripts to remove annoyances from webpages, and to do so i have to look at the html and the kind of trash you describe is already there.
Well here's the million dollar question. It's a maintenance dead end for humans to read and edit. But for an LLM, is it a maintenance deadend? Could the LLM iterate on that same code base and be highly effective on it?
Yes it is a maintenance deadend for LLMs it's notorious codebases start accumulating enormous amount of tech debt and that it gets almost impossible to unravel it even with the best agent
I've never seen it happen. You say this and it's likely one anecdotal claim. At the same time there's counter points like Bun getting rewritten in rust.
This is talk and talk is cheap. Prove it, otherwise it's still a million dollar question... unanswered.
HN is notoriously mentally deficient when it comes to AI. They were wrong about self driving cars (I sit in AI cars daily), they were wrong about AI getting used for coding (I don't use an IDE or type code anymore as a SWE). So I have to say unless there's something evidence based or substantial here it's likely given HN track record that most people here will end up being another wrong, baseless and over confident answer.
I'm looking for legit answers not confidently biased statements with no evidence.
1 reply →
[dead]
> I had an Iron Man moment
Iron Man created Jarvis whose capabilities are way beyond any models in the near future. So it wasn’t an Iron Man moment.
>> I had an Iron Man moment
> Iron Man created Jarvis whose capabilities are way beyond any models in the near future. So it wasn’t an Iron Man moment.
Like an LLM, you misunderstood the context. The voyeuristic experience doesn't require fiction to be reality.
He was presumably also not constructing a powered exoskeleton of from fictional materials or a physically implausible power source, but since you obviously caught the reference, how about some benevolent interpretation instead, for a decent shorthand about working smoothly with AI assistance.
(And on a personal note, I'm glad we don't have a publicly released Jarvis before we get our act together about the use.)