← Back to context

Comment by troupo

1 day ago

Running tests that the agent created for its own hallucinations? Optimize using another hallucinating agent?

Which tests and optimizations do you propose to run after a night of supervised work when one of main things that all agents keep doing is "load all records from db , and filter them in memory"? It's now become so bad, I had to literally vibecode a separate linter for this. And that's just one of the problems.

8 comments

troupo

Reply
serf  1 day ago

we don't have master AI that can create a great product out of a mediocre prompt.

but we do have sufficient AI to make a great product out of a great prompt.

garbage in -> garbage out hasn't gone anywhere.

so: much like to anyone that blindly complains that their compiler hates them : if you actually want help, provide information. If you just want to complain that the compiler is mean, scream at the sky.

plenty of people have figured out how to get this to work; more than enough to confirm that a straight <gambling-machine>/<hallucinatory-psychopath>/<random-number-generator> analogy is too simplistic to explain what we're working with.

  • troupo  1 day ago

    Demagoguery. Your "great product" (where is it? show me these great products) will have wildly ineficient code and things like "read the entire DB into memory" regardless of the prompt.

    > plenty of people have figured out how to get this to work

    Plenty of people claim they have figured it out. In reality these people are full of shit and assume that if LLMs can produce working software, it's great working software. And also assume that LoC is a measure of quality.

    Because without fail all the models keep doing this: https://news.ycombinator.com/item?id=48962703

    And you can only see that in your "great product" if you actually read the code and understand what's going on.

DonHopkins  1 day ago

>It's now become so bad, I had to literally vibecode a separate linter for this.

You see, there's your problem right there. You're vibe coding, which by definition literally means you're unwilling to look at the generated code. That's not what successful ai assisted software developers are doing. YOU HAVE TO READ THE CODE. Refusing to do that means you're not a serious programmer, you're outsourcing your thought and design and implementation, trying to get something for nothing by taking the easy way out, and you're going to get terrible results no matter what prompts you "engineer". There ain't no such thing as a free lunch (yet).

And while we're at it, to elaborate on what serf said: people mindlessly parroting terms like "stochastic parrot" to criticize llms without having read the actual paper that coined the term and understanding what it really claimed and how other papers responded to it means you're just a human stochastic parrot no better than what you're criticizing -- at least the llm has read all those papers and understands what "stochastic parrot" actually means in context. Ask it, it will be glad to explain!

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? (FAccT 2021)

https://dl.acm.org/doi/10.1145/3442188.3445922

  • troupo  1 day ago

    > You see, there's your problem right there. You're vibe coding, which by definition literally means you're unwilling to look at the generated code. That's not what successful ai assisted software developers are doing. YOU HAVE TO READ THE CODE.

    I guess you vibe-read what I wrote. Let me write it again for you: "I always have to correct its hallucinations during the day. Why would I ever let it run unsupervised overnight?"

    • DonHopkins  3 hours ago

      No, I read what you wrote, and you wrote "vibecoded", and the definition of vibe coding is not reading what you generated, then always hitting "Accept All". And it has a space in it. I'll quote the definition below, so you don't have to look it up yourself, and we can both be on the same page.

      It's right there on the label. If you didn't mean "vibe coded" then don't vibe write "vibecoded", and instead read what you wrote, then look up the meaning and spelling of the terms you are using, and do not neglect to correct yourself because it's not what you meant.

      I have to correct typos like that all the time when I write them by hand. But making typos and having to correct them and having to look up the meaning and spelling of words does not make me complain about all human generated code and text just because it requires me to proofread what I wrote.

      Reading what you wrote or what ai generated and looking up terms is all part of the process, because people and llms make mistakes. Vibe coding is not giving a shit about that, thinking it doesn't apply to you, and hitting "Accept All" without reading the code, by definition.

      And if you can't accept that, then don't write or generate code or text, and don't complain about how you have to read what you and llms generate and correct it, and understand what words mean.

      >I had to literally vibecode a separate linter for this

      Why would you "literally vibecode" a linter and use it to review other llm generated code without reading the linter's code itself? That is "literally" vibe coding eating its own tail.

      I take your emphatic use of the word "literally" to mean that you are "literally" using the well know definition of vibe coding as it was coined by Andrej Karpathy of OpenAI, which is "iterally" (and I quote):

      >I "Accept All" always, I don't read the diffs anymore.

      Am I misunderstanding you, or are you vibe writing the wrong term without reviewing the meaning of your own words?

      Here is the full literal quote so there is no room for confusion or claims of missing context:

      https://x.com/karpathy/status/1886192184808149383

      >Andrej Karpathy @karpathy: There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It's possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like "decrease the padding on the sidebar by half" because I'm too lazy to find it. I "Accept All" always, I don't read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it. The code grows beyond my usual comprehension, I'd have to really read through it for a while. Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until it goes away. It's not too bad for throwaway weekend projects, but still quite amusing. I'm building a project or webapp, but it's not really coding - I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works.

      That is the "literal" widely understood and well defined meaning of what you wrote, but misspelled "vibecoding", according to the well known AI expert from OpenAI who originally defined and championed the term. Its meaning has not suddenly changed.

      Only a vibe coder would vibe code a linter to vibe lint vibe coded code for them, without looking at ANY of that code themselves, by just hitting "Accept All" always. Because vibe coders by definition don't want to bother reading what they generated, don't care what it means, and still expect it to come out perfect.

      Don't be a vibe coder, or a vibe writer, or a human stochastic parrot: read what you write and know the definitions of the words you use.

      And if you're not a vibe coder, then don't claim to "vibecode": that's "not engineering" just "magical and wishful thinking", as you like to say.

      >"I always have to correct its hallucinations during the day. Why would I ever let it run unsupervised overnight?"

      And the answer to your question is: because you are using git, so you can look at the diffs before merging and deploying into production. You are using git and looking at the diffs, aren't you? Have you heard of PRs and code reviews? Or is that too much to ask of a "vibecoder"?

blackqueeriroh  1 day ago

If this is your problem, I hate to say it, but it’s a skill issue on your side. I didnt even start developing software until LLMs even though I know all the primitives quite well and have a strong grasp of architecture. ADHD, perfection, and focus got in the way. I’m writing a pretty complex application and I don’t have the problems you’re running into. I regularly have my code reviewed by professional devs to ensure I’m not just vibecoding into slop. I’m not. I also read my code regularly and do a lot of exercises and courses to keep learning as well.

So I dunno what to say, except it’s possible to write really solid code with LLMs.