Comment by stillpointlab
2 days ago
My advice is to start a totally vibe coded side project. You will never gain confidence until you experiment, find the failure cases, learn to deal with them, etc.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
I did this, but now I have a useful piece of software that I have zero clue regarding how it works. My only chance of "maintaining" it is giving it to another LLM to explain.
I am happy that I have a useful tool, but it simply isn't maintainable without a machine or me investing much time to understand it.
I don't see how the former is a sensible future for humanity. I am happy to say I won't be releasing this software I've created (I will find it useful myself), but deferring all understanding to a black box seems a poor future for brain activity.
One thing I do is ask the LLM to write me a walkthrough of the code. LLMs are pretty good at generating documentation like HTML slideshows or whatever else.
I mean, I've had useful software that I don't understand how it works where I am stuck hoping some small company continues to maintain it. And I've had some of those companies go out of business and leave me with useless software that I eventually have to replace once it no longer works on my machine.
Having the code that I can dump into an LLM to understand myself or have it alter it seems better than that.
It does, but then this admission that I have to feed the code into an LLM for it to generate documentation on how it works is just proving my point that we are now dependent on an LLM to understand the code that the LLM has generated.
It's the equivalent of promoting a sausage factory to be the solution to all of mankind's food problems. It's garbage.
Mermaid is a great output format for this type of information. Flowcharts, sequence diagrams, etc.
> One thing I do is ask the LLM to write me a walkthrough of the code.
I tried that. ChatGPT's result was laden with loud, clear and wrong fabications which I might have believed had I not examined the code.
Unreliable program be unreliable.
I've been working on a large (~10MLOC) mostly vibe-coded project for months now. It's actually quite frustrating:
* The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself. Yes, I can ask it to look elsewhere for similar bugs, but even that's not super reliable.
* The user-facing text it generates is just as awkward as ever. I have to rewrite all of it, and sometimes it slips through. It's also hard to write text when you don't know what the code is actually doing.
* It doesn't have a good model of what kind of information is relevant to humans. It's just as likely to produce a paragraph of random jargon as a clear explanation of what's actually going on. It shows excessive detail for things that don't matter, and not enough detail for things that do. For example, it doesn't understand that $0 can logically mean "null" in terms of how humans think about things (e.g. "This item is free" vs. "This item is $0").
I kind of wish I hadn't gone down this path but now it's far too late, I can't really manually maintain this amount of slop. I'm quite sad if this is going to be the end state of this profession.
> The agent will fix a bug when I tell it to, but only in exactly one place. I'll notice that another place has that bug later and have to redo it. It's the sort of thing that I would know to check for if I were fixing it myself.
I see the same in e.g. simple CSS. I ask ChatGPT to apply styling X to aĺl instances of say a UI control type. Then to another. Later I ask for a tweak of X. Bot updates some but not all instances. It didn't create a single class for all. No intelligence seen.
The most remarkable fails are simpler though. E.g. at version 303 I ask "Revert to version 299". I get 299 with 303's edits. Bot explains it interpreted "Revert" to mean something no intelligent programmer would.
And most importantly, these are just the ones I directly notice. To see the total, I have to read through and understand the most repulsive code I've seen in my life.
The only way to be satisfied with bot code is by having exceptionally low expectations. And never ever let yourself be kidded that your helper is more than a stochastic parrot gorged on stackoverflow etc.
Ridiculous suggestion. If someone already takes issue with ai generated code (as well they should, it often is _still_ pretty bad), getting them to write more of it is useless.
Additionally, do you really believe that 'Now is the cheapest time it will ever be to experiment'? Why so? Because of token prices? Not a good omen if so.
I'm not sure what is ridiculous about suggesting people take a stance of openness and curiousness when confronted with new situations. I genuinely don't understand what people have to lose by building some throw-away experiments without reading the code.
I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
>I do believe the subsidies offered on Pro accounts driven by the current competition make access to the best coding models and harnesses as cheap as they will ever be.
God I hope you're right. Maybe then this era of vibe slopping out photoshop clones that'll be abandoned within a week (while advertising them all over social media) will come to a close.
> 'Now is the cheapest time it will ever be to experiment'?
First hit is free ;)
I've done this, and always get subpar results. I cant get the LLM to make the minor tweaks I want to see, and now I have no mental model of the code to make the fix myself. Sometimes they arent minor things.
I have done this. It's exactly the reason I went back to doing code review. I don't look at the code with a fine toothed comb, but it does help me catch bad patterns early.
I guess the question is: did the project you worked on fail? As in, you couldn't achieve the outcome you were trying to achieve? Or at some point did you sneak a look at the code and recoil in revulsion? What was the failure state?
Was this with the latest models (Fable/Opus 5.5 or equivalent) or an older model in a 3rd party harness?
Are you in the camp of: "if at first you don't succeed, never ever try again".
Not parent, but project I was involved in was shipped very quickly, had many bugs, and burned a lot of customer trust.
This was a few months ago, so obviously not with the latest models, but:
1. That argument is evergreen. You will always be able to make it, and it doesn’t really add much. It is impossible to refute.
2. Gone back to - when writing code - having a strict mental modal and good control over the codebase. A mix of hand coding (to establish the mental modal) and prompting (for boilerplate). It’s not quite as fast, but the quality is much higher when you mix in using LLMs in quality enhancing capacities (review, debug, etc). This is with the latest model. Hands off still atrophies over time (even the boilerplate), and in some cases is very inefficient (as in wasting my time, nothing to be said about tokens).
> Are you in the camp of: "if at first you don't succeed, never ever try again".
This is great in some domains, but “don’t make the same mistake twice” is much more important in mine. I don’t have the luxury to ship low quality, especially when LLMs make even higher quality so much easier.
1 reply →
[flagged]
Please don't do this here.
(You're welcome to make your substantive points thoughtfully, of course.)
https://news.ycombinator.com/newsguidelines.html
[flagged]
Issue: I am still unable to stop myself from doing line-by-line code review
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
The thing is that's not an issue in the slightest, it doesn't need therapy!!
It's those who don't do it who'd need a talk..!
4 replies →
No. The issue is: "I notice too many things (redundant code, duplicate code, overly complex architecture, etc.) during review"
How does "low-stakes environment" fix that?
9 replies →
I think the intended takeaway is to learn to solve those problems without line by line review. Start by being hands off on a low stakes project, and see how the model fails. Then, come up with, and try, solutions that aren't you doing archeology.
This somehow assumes that the person above doesn't already do all the magic the people say work 100% of the time, and finds these issues. How does running into literally the same issues in a low-stakes project help?
4 replies →