Comment by wongarsu
6 hours ago
Doesn't the Linux kernel encourage reporting on the public mailing list for kernel vulnerabilities that were found using AI? If I remember correctly they introduced that policy after the security mailing list got swamped with multiple reports for the same vulnerabilities
On first glance it seems you are right.
> If you resorted to AI assistance to identify a bug, you must treat it as public. https://github.com/torvalds/linux/blob/master/Documentation/...
It is very interesting that they believe AI will find different bugs than people. That is, if you found a bug with AI the expectation is that others could find the same bug easily. But if you find it yourself, it is somehow more of a sensitive secret.
I do not imagine that things divide so clearly along these lines.
I think the expectation is that if you found it with an LLM them other people are also going to find it with an LLM, so it's not very private anyway. That's not the same as saying if you find it with an LLM it will not be found by manual review, or vice versa.
I think this idea comes from two directions: firstly, LLMs are relatively predictable, so given the same prompts and review processes you're likely to find the same bugs, and secondly, if you're using the free or cheaper LLM tiers, your conversation is likely to end up in the training data and so become 'known' the the LLM after some time anyway. I think the latter is perhaps being overestimated by the kernel security team (Greg KH's talk on the matter characterised is as "the LLM finds a bug for one person and then decides to hand it over anyone else looking for bugs as well", which really isn't how it works internally).
Probably not a clear divide, but it might be some kind of divide, and it might be the opposite of intuition.
The big advantage the LLMs have is their huge context windows, ability to parallelize and delegate, and no attention fatigue or need to sleep. They can think about random combinations of functions and modules all day long until they find a combination that's exploitable. That's hard for a person to do, and its the kind of exploit that probably would never be found otherwise except by another llm-powered thing. Not being a security expert myself, I gather that these AI vulns tend to be small needles in very large haystacks that normal people just would never have the patience to search.
But if you found something yourself, then it's something that's simple enough and involves a small enough number of moving parts that you can keep it in your head all at once and recognize the problem and correctly reason about its consequneces, which probably means it's easier to exploit, and therefore more dangerous if made public.
It feels a little backwards, and it might still be backwards, but if there's any benefit i think it's along those lines.
The policy is based on what is actually occurring. The Linux security team has had a flood of duplicate reports from AI, so they changed their policy to request that security bugs found by AI be reported on public mailing lists to avoid duplicates. They have not yet seen the same for human found bugs, so the old policy remains in place. It is likely that at some point in the future the AI discovered bugs will be a superset of human discovered ones, but for now humans and AI seem to be better at finding different types of bugs. When/if it does happen, the number of human bugs will taper off to nothing because AI will find them all first, so the policy change will take care of itself.
It's a shared tool so it coming to similar conclusions across users wouldn't be unexpected. Like if it were a scanner you could run to find common security problems, you would treat those as public since anyone could just run the scanner.
You are misrepresenting the premise. The premise is "if you found a bug with a COTS toolkit, others using the same toolkit may also find this bug easily". The second assumption is that many people are using the same COTS toolkit, which seems uncontested in my eyes as there's only a handful models in wide circulation.
I agree with this comment. Feels to me like they thought every prompt was "find the bug here," but in reality a lot of people put interesting context or original thoughts into their workflow with LLMs.
“If you resorted to AI assistance to identify a bug, you must treat it as public. While you may have valid reasons to believe it is not, the security team's experience shows that bugs discovered this way systematically surface simultaneously across multiple researchers, often on the same day. In this case, do not publicly share a reproducer, as this could cause unintended harm; just mention that one is available and maintainers might ask for it privately if they need it.”
Do not publicly share a reproducer aka ai found bug
I don't think AI is of much concern, if you actually have a valid security issue.
No-one, really will care if you discovered it via AI or otherwise. Because, the weight of the bug itself should outweight other concerns. This is how professional security works. There will always be one concern or the other about a bug but what carries more weight will determine the direction that the disclosure process takes.
A valid security bug, by itself should carry enough technical weight that concerns about whether it was discovered through AI or otherwise should become irrelevant.
Anyways, it mostly seems like people don't really care much about computer security at Linux. I would like to think they care more about infrastructure and related security issues because, well, if my computer is an open book, then why would security issues around it be a concern?
Anyways...
Enough said here - lol
>A valid security bug, by itself should carry enough technical weight that concerns about whether it was discovered through AI or otherwise should become irrelevant.
that's not really what the policy is about.
the thinking behind the policy is that if you found it via AI, other people can trivially find it via AI. that is why AI-found bugs are treated as public from the start.
4 replies →
As it should. If an LLM discovered them you should assume that another LLM, or another person running the same LLM, could find them. They are effectively public already.
No. I have an LLM and didn't discover it. Folks in HR have LLMs but can't produce the same software you can. Just because an LLM could discover it does not mean that it has been discovered yet. Responsible disclosure is still a valuable thing.
>I have an LLM and didn't discover it.
are you actively pointing your LLM at the kernel to identify security bugs?
the policy wasn't made on a whim. to quote the linux security team: "bugs discovered this way [via ai] systematically surface simultaneously across multiple researchers, often on the same day."
>Responsible disclosure
the term responsible disclosure is a silly thing made up by vendors.
2 replies →