Hard to say. Claude’s very good at writing READMEs. In fact, Copilot often complains about docs that sound like they’re about current capabilities when in fact they’re future plans or just plan aspirational.
Without downloading and testing out your software, how can we know if it’s any good? Why would we do that if it’s obviously vibed? The dilemma.
I’m not at all against vibe coding. I’m just pointing out that having a nice README is trivial. And the burden of proof is on you.
I’m just curious, what would need to happen for you to change your opinion about this? Are you basically of the opinion that it’s not good enough today, never will be good enough in the future, and we should just wind back the clock 3 years and pretend these tools don’t exist?
It feels to me like a lot of this is dogma. If the code is broken or needs more testing, that can be solved. But it’s orthogonal: the LLM can be used to implement the unit testing and fuzz testing that would beat this library into shape, if it’s not already there. It’s not about adding a human touch, it’s about pursuing completeness. And that’s true for all new projects going from zero to one, you have to ask yourself whether the author drove it to completeness or not. That’s always been true.
You want people to hedge their projects with disclaimers that it probably sucks and isn’t production worthy. You want them to fess up to the fact that they cheated, or something. But they’re giving it away for free! You can just not use it if you don’t want to! They owe you nothing, not even a note in the readme. And you don’t deserve more or less hacker points depending on whether you used a tool to generate the code or whether you wrote it by hand, because hacker points don’t exist, because the value of all of this is (and always will be) subjective.
To the extent that the modern tools and models can’t oneshot anything, they’re going to keep improving. And it doesn’t seem to me like there’s any identifiable binary event on the horizon that would make you change your mind about this. You’re just against LLMs, and that’s the way it is, and there’s nothing that anyone can do to change your mind?
I mean this in the nicest way possible: the world is just going to move on without you.
Sorry but these are just not accurate as blanket statements anymore, given how good the models have gotten.
As other similar projects have pointed out, if you have a good test suite and a way for the model to validate its correctness, you can get very good results. And you can continue to iterate, optimize, code review, etc.
AI often produces nonsense that a human wouldn't. If a project was written using AI the chances that it is a useless mess are significantly higher than if it was written by a human.
I work on a revision control system project, except merge is CRDT. On Feb 22 there was a server break-in (I did not keep unencrypted sources on the client, server login was YubiKey only, but that is not 100% guarantee). I reported break-in to my Telegram channel that day.
I used tree-sitter for coarse AST. Some key parts were missing from the server as well, because I expected problems (had lots of adventures in East Asia, evil maids, various other incidents on a regular basis).
When I saw "tree-sitter in go" title, I was very glad at first. Solves some problems for me. Then I saw the full picture.
Wait, are you suggesting that OP broke in to your server and stole code and is republishing it as these repos?
I have questions. Have you reviewed the code here to see if it matches? What, more specifically, do you mean when you say someone broke in? What makes you think that this idea (which is nice but not novel) is worth stealing? If that sounds snarky, it’s not meant to; just trying to understand what’s going on. Why is that more likely than someone using Claude to vibe up some software along the same lines?
I can't speak for the specificity of parent's "evil maids" phrase but the concept of an "Evil maid" is used in security scenarios.
A maid tends to be an example of a person who's mostly a stranger, but is given unmonitored access to your most private spaces for prolonged periods of time. So they theoretically become a good vector for a malicious actor to say "hey I'll give you $$ if you just plug in this USB drive in his bedroom laptop next time you're cleaning his house" - it's often used in the scenario of "ok what if someone has physical access to your resource for a prolonged period of time without you noticing? what are your protections there?"
This was my first thought as well, just from reading the title.
well how did it do?
Hard to say. Claude’s very good at writing READMEs. In fact, Copilot often complains about docs that sound like they’re about current capabilities when in fact they’re future plans or just plan aspirational.
Without downloading and testing out your software, how can we know if it’s any good? Why would we do that if it’s obviously vibed? The dilemma.
I’m not at all against vibe coding. I’m just pointing out that having a nice README is trivial. And the burden of proof is on you.
Shouldn't you be able to answer that?
yes and if you clicked the links you would know that i did answer it in the readme.
3 replies →
How is OP using Claude relevant?
OK for prototyping. Not OK for prod use if noone actually read it line by line.
I’m just curious, what would need to happen for you to change your opinion about this? Are you basically of the opinion that it’s not good enough today, never will be good enough in the future, and we should just wind back the clock 3 years and pretend these tools don’t exist?
It feels to me like a lot of this is dogma. If the code is broken or needs more testing, that can be solved. But it’s orthogonal: the LLM can be used to implement the unit testing and fuzz testing that would beat this library into shape, if it’s not already there. It’s not about adding a human touch, it’s about pursuing completeness. And that’s true for all new projects going from zero to one, you have to ask yourself whether the author drove it to completeness or not. That’s always been true.
You want people to hedge their projects with disclaimers that it probably sucks and isn’t production worthy. You want them to fess up to the fact that they cheated, or something. But they’re giving it away for free! You can just not use it if you don’t want to! They owe you nothing, not even a note in the readme. And you don’t deserve more or less hacker points depending on whether you used a tool to generate the code or whether you wrote it by hand, because hacker points don’t exist, because the value of all of this is (and always will be) subjective.
To the extent that the modern tools and models can’t oneshot anything, they’re going to keep improving. And it doesn’t seem to me like there’s any identifiable binary event on the horizon that would make you change your mind about this. You’re just against LLMs, and that’s the way it is, and there’s nothing that anyone can do to change your mind?
I mean this in the nicest way possible: the world is just going to move on without you.
8 replies →
ii am trying to not take issue with this comment because im aware of the huge stigma around ai generated code.
i needed this project so i made it for my use case and had to build on top of it. the only way to ensure quality is to read it all line by line.
if you give me code that you yourself have not reviewed i will not review it for you.
That ship has sailed, man…
1 reply →
Sorry but these are just not accurate as blanket statements anymore, given how good the models have gotten.
As other similar projects have pointed out, if you have a good test suite and a way for the model to validate its correctness, you can get very good results. And you can continue to iterate, optimize, code review, etc.
People should say what models/tools they used in even show the prompts.
"show the prompts"
What would the prompt for this look like?
Because the entire README doesn't even mention it, and it is an important factor in deciding whether it is ready for production use.
I, for one, am definitely not going to use this project for anything serious unless I have thoroughly reviewed the code myself. Prototyping is fine.
Because OP obviously downplayed this important fact, which typically shows lower quality/less tested code.
maintenance burden
AI often produces nonsense that a human wouldn't. If a project was written using AI the chances that it is a useless mess are significantly higher than if it was written by a human.
I work on a revision control system project, except merge is CRDT. On Feb 22 there was a server break-in (I did not keep unencrypted sources on the client, server login was YubiKey only, but that is not 100% guarantee). I reported break-in to my Telegram channel that day.
My design docs https://replicated.wiki/blog/partII.html
I used tree-sitter for coarse AST. Some key parts were missing from the server as well, because I expected problems (had lots of adventures in East Asia, evil maids, various other incidents on a regular basis).
When I saw "tree-sitter in go" title, I was very glad at first. Solves some problems for me. Then I saw the full picture.
Wait, are you suggesting that OP broke in to your server and stole code and is republishing it as these repos?
I have questions. Have you reviewed the code here to see if it matches? What, more specifically, do you mean when you say someone broke in? What makes you think that this idea (which is nice but not novel) is worth stealing? If that sounds snarky, it’s not meant to; just trying to understand what’s going on. Why is that more likely than someone using Claude to vibe up some software along the same lines?
1. Just saying, strange coincidence
2. How can we compare Claude's output in a different language?
3. Detecting break-ins and handling evil-maids: unless the trick is already known on the internets, I do not disclose. Odds are not in my favor.
4. Maybe worth, maybe not. I have my adaptations. Trying to make it not worthy of stealing, in fact.
6 replies →
LMFAO what is this
Also, evil maids, what?
I can't speak for the specificity of parent's "evil maids" phrase but the concept of an "Evil maid" is used in security scenarios.
A maid tends to be an example of a person who's mostly a stranger, but is given unmonitored access to your most private spaces for prolonged periods of time. So they theoretically become a good vector for a malicious actor to say "hey I'll give you $$ if you just plug in this USB drive in his bedroom laptop next time you're cleaning his house" - it's often used in the scenario of "ok what if someone has physical access to your resource for a prolonged period of time without you noticing? what are your protections there?"
I wonder if that's what OP meant? :-)
3 replies →