Comment by tyre
8 hours ago
Does it matter? They found 12 vulnerabilities. Clearly there was enough signal:noise that they could uncover these as real.
It doesn't look like they had 1 AI run for 20 minutes and then 30 humans sift through for weeks.
8 hours ago
Does it matter? They found 12 vulnerabilities. Clearly there was enough signal:noise that they could uncover these as real.
It doesn't look like they had 1 AI run for 20 minutes and then 30 humans sift through for weeks.
Does it matter?
Yes, we have been on the receiving end of AI generated bug reports and in the vast majority of cases they are really bad. But you still need humans to sift through them. And when you ask the submitter questions, it’s often clear that they just give the questions to an LLM again to answer.
It costs a huge amount of human manpower, so if the company who made this had an AI based solution with a far lower false-positive rate, that would be great.
> It doesn't look like they had 1 AI run for 20 minutes and then 30 humans sift through for weeks.
It does, though, look like they were running their AI over the codebase for an extended period of time (not per run, but multiple runs over the period of a year)
> Does it matter?
Hell yes, false reports are the bane of the bug bounty industry.