Comment by zeroonetwothree
2 days ago
I've been working on a large (~10MLOC) mostly vibe-coded project for months now. It's actually quite frustrating:
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
> The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself.
I see the same in e.g. simple CSS. I ask ChatGPT to apply styling X to aĺl instances of say a UI control type. Then to another. Later I ask for a tweak of X. Bot updates some but not all instances. It didn't create a single class for all. No intelligence seen.
The most remarkable fails are simpler though. E.g. at version 303 I ask "Revert to version 299". I get 299 with 303's edits. Bot explains it interpreted "Revert" to mean something no intelligent programmer would.
And most importantly, these are just the ones I directly notice. To see the total, I have to read through and understand the most repulsive code I've seen in my life.
The only way to be satisfied with bot code is by having exceptionally low expectations. And never ever let yourself be kidded that your helper is more than a stochastic parrot gorged on stackoverflow etc.