No, he's still dealing with a flood of crap, even in the last few weeks, off more modern models.
It's primarily from people just throwing source code at an LLM, asking it to find a vulnerability, and reporting it as-read, without having any actual understanding of if it is or isn't a vulnerability.
The difference in this particular case is it's someone who is:
1) Using tools specifically designed for security audits and investigations.
2) Takes the time to read and understand the vulnerability reported, and verifies that it is actually a vulnerability before reporting.
Point 2 is the most significant bar that people are woefully failing to meet and wasting a terrific amount of his time. The one that got shared from a couple of weeks ago https://hackerone.com/reports/3340109 didn't even call curl. It was straight up hallucination.
I think it's more about how people are using it. An amateur who spams him with GPT-5-Codex produced bug reports is still a waste of his time. Here a professional ran the tools and then applied their own judgement before sending the results to the curl maintainers.
I keep irritating people with this observation but this was the status quo ante before AI, and at least an AI slop report shows clear intent; you can ban those submitters without even a glance at anything else they send.
It's probably also the difference of idiots hoping to cash out/get credit for vulnerabilities by just throwing ChatGPT at the wall compared to this where it seems a somewhat seasoned researcher is trialing more customized tools.
No, he's still dealing with a flood of crap, even in the last few weeks, off more modern models.
It's primarily from people just throwing source code at an LLM, asking it to find a vulnerability, and reporting it as-read, without having any actual understanding of if it is or isn't a vulnerability.
The difference in this particular case is it's someone who is: 1) Using tools specifically designed for security audits and investigations. 2) Takes the time to read and understand the vulnerability reported, and verifies that it is actually a vulnerability before reporting.
Point 2 is the most significant bar that people are woefully failing to meet and wasting a terrific amount of his time. The one that got shared from a couple of weeks ago https://hackerone.com/reports/3340109 didn't even call curl. It was straight up hallucination.
I think it's more about how people are using it. An amateur who spams him with GPT-5-Codex produced bug reports is still a waste of his time. Here a professional ran the tools and then applied their own judgement before sending the results to the curl maintainers.
I keep irritating people with this observation but this was the status quo ante before AI, and at least an AI slop report shows clear intent; you can ban those submitters without even a glance at anything else they send.
The current scale of poor reports was absolutely not the status quo before AI
1 reply →
It's probably also the difference of idiots hoping to cash out/get credit for vulnerabilities by just throwing ChatGPT at the wall compared to this where it seems a somewhat seasoned researcher is trialing more customized tools.