Comment by consumer451
11 hours ago
Sorry for the meta, but the discourse in many of the comments is nuts to me.
"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?
Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.
edit: The giant thread with this now got flagged, so some users may not see it.
I get you, but ai is different. Me using hand tools to fashion a custom piece of furniture is using tools in an impressive way. While o didn’t find this bug with AI, it’s obvious that AI is a tool altogether different. Yes there is agency and some degree of creativity/finesse in getting it to do what you want, but it does all the heavy lifting. All of it. You just have to poke it and prod it and rub it just right to get the magic gene out of the lamp. But that is such a far cry from knowing what’s needs to be known to do the task yourself, not to mention discovering its feasibility. It’s just wildly different than any tool we’ve known. So obviously people will snark when people don’t give clear and explicit credit.
Would Claude have found it without your guidance and knowledge? If yes, Claude found it. If no, you found it while using an LLM.
I suspect a lot of things fall into the latter. It is a common neg to defer everything to the former though.
If we're doing this standard, then let's do it for everything. Claude coded that, not you. Claude built that website, not you. Might make the industry less exhausting tbh.
GPT did the Hugging Face hack, not you (human engineers at OpenAI) :^)
The best response to them is "If it's so easy, then when didn't you do it? Easy $90k."
One day soon, this will be like people saying you used a high level language instead of assembly to discover a fault.
People who believe AI is an abstraction layer on top of compilers, are in that middle ground where they know computer science but are far from an expert. Which makes them very susceptible to the Dunning–Kruger effect. Throw in AI sycophantcy and you've got yourself takes like the one above.
Ha.. AI in the form of LLM is not a very good non-deterministic abstraction layer on top of deterministic compilers.
Yet, they are being used with endless surprise as non-deterministic layers of abstraction on top of deterministic compilers.
My point is to use a technology for what it is capable for (where capabilities are discovered weekly and hard to keep up with), not trying to get it to do everything.
Bringing a deterministic lens to non-deterministic software has its dangers too. Non-deterministic software seems to be its own category of software, increasingly capable of non-deterministic approaches that deterministic compilers software couldn’t attempt.
Maybe that’s less compilers and more different type of scaffolding of vectors.
I’m all for skepticism, the issue with cynicism is capabilities often pass those perspectives by sometimes too quickly.
1 reply →
Yeah some people come off as envious they didn't have Claude try it first.
But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.
"Claude found a $90K RCE" would have been just as interesting if not more.
Sure, but do we want traditional vuln posts to be “nmap found”?
It is fair an interesting to disclose tools used, but I am with GP in leaving agency with humans, at least for now.
Nmap doesn't write its own blog posts. That's not where I draw the line, but the point isn't at all about "drawing the line." (In other words, don't get caught up in where the line is. There is one, but we're not sitting in some gray area, we've blown past it by hundreds of kilometers by now.)
If I post an image and I say, "I drew this", and someone says, "no you didn't. That is clearly output from Qwen Image 2.1", can I reply, "Qwen Image 2.1 is just a tool, like Photoshop"? I mean, I can, but if I do this, I am missing the point. The point is that I didn't draw anything. I didn't draw. I didn't draw digitally. I didn't draw analog. I didn't draw with my mind. I typed a prompt. This is as close to drawing as photography is to drawing.
So did this person actually find a vulnerability on their own? Hell if I know, but I take issue with LLM-authored blog posts; I don't trust people who post LLM-authored blog posts as if they were written by themselves to be honest about the nature of their work.
If you really are still doing work and understanding and just using LLMs as a tool, show your understanding in your blog post that you are communicating to the world, instead of having Claude or GPT show its understanding. I don't really care whether you used Claude and didn't list it, if it's clear as day you understand what you were doing with it.
I'm dying on this hill. Until HN bans this sort of blog post or bans all of us complaining, these comments are going to be very, very boring. And I am stubborn for longer than most people can stand to read the same comments. Believe me.
3 replies →