BTW regarding "suggested fixes", an interesting attack would be to report a bug along with a prompt injection which will cause Claude to suggest inserting a vulnerability in the codebase in question. So, it's important to review bug-report-originated Claude suggestions extra carefully. (And watch for prompt injection attacks.)
Another thought is the reproducible builds become more valuable than ever, because it actually becomes feasible for lots and lots of devs to scan the entire codebase for vulns using an LLM and then verify reproducibility.
No. I would probably do something like: Have Claude Code replicate + triage everything. If a report gets triaged as "won't fix", send an email to the reporter explaining what Claude found and why it was marked as "won't fix". Tell the reporter they still have a chance at the bounty if they think Claude made a mistake, but they have to pay a $10 review fee to have a human take a look. (Or a $1 LLM token fee for Claude to take another look, in case of simple confabulation.)
Note I haven't actually tried Claude Code (not coding due to chronic illness), so I'm mostly extrapolating based on HN discussion etc.
BTW regarding "suggested fixes", an interesting attack would be to report a bug along with a prompt injection which will cause Claude to suggest inserting a vulnerability in the codebase in question. So, it's important to review bug-report-originated Claude suggestions extra carefully. (And watch for prompt injection attacks.)
Another thought is the reproducible builds become more valuable than ever, because it actually becomes feasible for lots and lots of devs to scan the entire codebase for vulns using an LLM and then verify reproducibility.
Would you ever blindly trust it?
No. I would probably do something like: Have Claude Code replicate + triage everything. If a report gets triaged as "won't fix", send an email to the reporter explaining what Claude found and why it was marked as "won't fix". Tell the reporter they still have a chance at the bounty if they think Claude made a mistake, but they have to pay a $10 review fee to have a human take a look. (Or a $1 LLM token fee for Claude to take another look, in case of simple confabulation.)
Note I haven't actually tried Claude Code (not coding due to chronic illness), so I'm mostly extrapolating based on HN discussion etc.