Comment by ChrisMarshallNY
10 hours ago
The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult.
But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.
The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.
This is what I truly don't like about the CVE system.
One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It was so unused that the solution for zlib was to simply remove it.
I know about this one particularly because our security policy required us to do a BUNCH of pointless updates for it since zlib is in just about everything.
[1] https://app.opencve.io/cve/CVE-2023-45853
In the Spack package manager [1] we're trying to work around this with directives:
So, you can define conditional deprecation, and the dependency resolver can still pick up `zlib ~minizip` (i.e. with a variant/component disabled).
Deprecation on versions only isn't the right granularity.
[1]: https://github.com/spack/spack/pull/52372
Also a buffer overflow in user space for a tool not serving internet traffic is not a big deal.
2 replies →
Reminds me of the time our “security” team fought with me for a year to fix an issue in a third-party dependency that was failing their scanners. The issue was only a problem for and intentionally left in to not break support for Internet Explorer 6 on Windows XP, neither of which anyone at the company had used in 10 years, and everyone using this software was on a Mac anyway.
My condolences to anyone who’s got to deal with all these slop-y CVEs on one side and brain dead security teams on the other.
same things happens with npm and CVEs, most of the ones that bubble up aren't even called in my application so are no applicable.
go lang has a scanner [0] which only shows issues when the code actually calls the effected functions.
[0] https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck
IMHO that’s why binaries and libraries should be separate packages. Drives me buts with npm packages.
7 replies →
But could an attacker with access to zip tools exploit it to get a root shell? Probably not, but maybe.
8 replies →
Isn't the real difficulty in how vulnerabilities can be mixed? A given vulnerability might be extremely hard to execute, but very damaging. Another vulnerability might be easier to execute, but it can't do much. But if the second one can be used to trigger the first one, you then have attack surface area of the second vulnerability with the damage of the first one.
Even if some individual case can be shown to be safe from being combined, can we identify such cases with enough confidence to justify using it reduce severity warnings?
1 reply →
reminds me of the cvss scoring system. that thing is great at telling a story. for better or worse XD.
If the solution was to remove an unused dependency, I think this CVE isn't silly at all.
Yes, the CVE system is broken. LLM-generated CVEs is making it even worse.
I suppose all these fake issues and the many more that have absurdly elevated severities could be considered an attack on the system itself, stripping it of credibility.
That's the worst case scenario right there: so many fake reports that the real vulnerabilities can no longer be reported properly and fixed quickly. Assuming some LLMs really are capable of finding serious vulnerabilities (which in my experience is absolutely the case : just let a good model run on your security code base and see for yourself) that means that we can more easily find vulnerabilities, but it's much harder to report them and be taken seriously - so the incentive to exploit such vulnerabilities instead has become much, much higher.
1 reply →
Yes. This was the case long before LLMs sprayed the tire fire with gasoline. See e.g. this 2021 gem from Dan Abramov (React's creator) https://overreacted.io/npm-audit-broken-by-design/
CVE-2026-4I5109 - Your kitchen drawer contains forks, which can maim and/or kill. An attacker with kitchen access could access the drawer. See...
CVE-1972-404: The code is written in C. ^_^
The majority these days seems to be real issues, but probably not exploitable. The latest LLMs are very good at reading code and finding issues where the code is wrong. They are somewhat less good at finding exploits - both because finding exploits is harder than finding code that is clearly wrong (even if not exploitable), and also because the better ones are intentionally made to not create exploits (since they don't know if you are a good or bad - they assume bad.
However if I was writing this response just one year ago I would instead be saying: the majority off LLM CVS are noise where the code is correct, and often they are writing up for code that doesn't even exist.
Which is to say I suspect the repo in question was generated with a year-old LLM, since they act like that. The new ones [mostly?] are much better.
Still, if a modern LLM points out something you should fix it. Even if we can't figure out how to exploit it today that doesn't mean we won't figure it out in the future.
> I suspect the repo in question was generated with a year-old LLM
I suspect there are a lot of people running inexpensive models that are searching for vulnerabilities across a lot of projects, probably in an automated way (ex with openclaw or similar) in the hopes of winning a bounty and/or noteriety.
1 reply →
> The vast majority of CVEs are not exploitable, basically noise.
This is not true if you consider security-in-depth. Many of them are exploitable on their own but maybe not in combination with other issues that are as yet unknown or known but not patched everywhere.
As a simple example a local privilege escalation issue that is not exploitable on a device that only I ever have access to, essentially becomes a remote root access flaw if you have untrusted or unreliable users (clients with accounts for instance) on the system. This works on a finer grain too, seemingly minor issues spread through the kernel and user space can add up to a serious exploit.
This is the most common "not exploitable" CVE I have to deal with at work all the time. Things like buffer overruns when passing headers, but my server shouldn't be accessible to the public Internet anyway, and if it is, then we have much bigger problems. So yes it's technically exploitable IF other safeguards fail, but the reason it's irrelevant is that if those other safeguards fail, we are in deep shit anyway.
4 replies →
I once had a customer threaten to reject a delivery over a CVE. I dug into it. The CVE was that an Ocaml PostgreSQL client didn’t implement an authentication feature.
I convinced the customer to accept the delivery by pointing out that (1) our app had zero lines of ocaml and (2) the feature had been implemented in the ocaml driver since the CVE was issued.
>The vast majority of CVEs are not exploitable
Perhaps in isolation. The issue is when you can chain exploits to bypass multiple layers.
Not exploitable, or not exploitable in isolation?
Are you trying to say LLM’s are tuned to only discover unexploitable bugs?
I'm sending all my condolences out to everyone who is in a corporate environment using tools like Veracode or Snyk to automatically assess for CVEs and block merges or deployments based on "findings".
My experience in such environments leads me to believe this is going to be a rough ride for those heavily locked-down enterprises, because depending on the environment, an exception of "this CVE was hallucinated by AI" is probably going to be difficult to get accepted, and when it does, starts to become its own avenue for exploitation and adds even more noise and confusion to the mix.
They’re not bad if you give up on being able to reproduce arcane bugs. Don’t pin versions, pull everything from latest, CVEs are just a rebuild. Certainly not an amazing idea, but it solves the problem of the day.
The funnier, Kafakaesque problem of the day is interactions with mandatory cooldown periods on new versions because of supply chain risks.
I’ve had a couple tickets get stuck because the CVE scanner says I have to update, but the cooldown enforcer says the version hasn’t been out long enough.
>Don’t pin versions
In enterprise software..... I only dream of work where some ancient version isn't pinned because 200 person hours are need to adapt the code to newer versions.
And the update versus cooldown are fun. Can't win either way you go.
> Don't pin versions, pull everything from latest
Rebuild broke half your tests. People are exploiting the CVE against your system right now. Good luck!
2 replies →
In our organisation, my security team spend the majority of our effort weeding out unexploitable CVEs - which are the majority of them. But the volume is growing year on year - I don't know how sustainable it's going to be.
Sounds like a good job for AI. I do fear that the future will involve deploying AI do deal with the mountain of slop produced by other AI's.
1 reply →
Snyk routinely tells us that we need to upgrade urgently because they found a vulnerability in the Windows version of a package. We run everything in Linux.
[dead]
CVEs sucked even before LLMs. CVSS is an ass-covering exercise, not an importance signal. Everything is "critical" all the time.
Which is precisely why I do not celebrate when Microsoft or other companies say they released "their biggest set of security vulnerability fixes ever" because there's a good chance that most of them are hallucinations or non-critical issues erroneously considered zero-days or critical.
What actually needs to be done is human review, vetting, and approval of those. Otherwise, it's just unknown/unnecessary/potentially dangerous changesets in high quantities.
Eh, when we see this from Microsoft, Linux kernel, and Apple all at the same time just thinking it's everybody being dumb becomes risky.
Now, if you're an F100 that's currently not using Anthropic it's pretty likely you can go to them now and ask for a non-guardrailed unlimited Mythos trial for a month you can get an agent that will not only find bugs for you, but also write exploits in a framework that will demonstrate them in action. Once you see an AI chain a bunch of shit together, get a foothold and then expand around laterally in your system you start getting concerned about how you're going to keep up.
The time has come to pay for our sins of shit security.
> I do know that LLMs have been discovering a lot of legit CVEs,
And that's great! Now the human in the loop can verify the accuracy of each finding and take responsibility for moving it forward.
> and I will lay odds that the blackhats are leveraging them to the max.
We just need more humans in the loop to do that more quickly, each putting their own reputation on the line with each CVE they determine to be legit.
AI doesn't stand for "accountability irrelevant", even though most people act like it does.
We can shoot for 100% accountability by demanding 100% accuracy.
Of course what will really happen is anyone looking for problems will stop going to the source for fixes and will start gaining cred in the dark markets of exploits and problems, which is better for everyone, right?
Just because someone is using AI doesn't mean they didn't find a bug, it's more likely they just have no means of determining if it's a real bug or not, so to the maintainers tickets seem random. The problem comes in when clever attackers can take these untrustworthy tickets and run them through another LLM that determines if an exploit is possible and then uses the attacks it finds before anyone has a chance to patch.
Basically nothing is real nowadays unless you verify it for yourself.
This is an issue with CVEs not LLMs
I think in both cases, the LLM use needs to be handled by someone experienced with the domain. We are seeing this over and over with product development, writing, images, etc. Without good people driving this it's just noise
[flagged]
[dead]
Edits for clarity: seems like this was a controversial take, which I find pretty interesting.
I believe people took this comment as LLMs being better than security aware engineers who have the time to spend building solid systems.
This wasn't the point, the reality outside "established" tech companies is that software security can be lackluster.
There often simply isn't enough resources to check old software for basic vulnerabilities, outdated packages with known issues, there might be a manager who insists on a certain solution. Or that certain services are on "maintenance mode", but rarely get checked since they're on a certain part of the internal network.
If they are able to properly scan their full software stack for CVE issues, they cannot deal with a flood of CVEs.
----
We're in a transition period where AI will eventually make software much more secure than it ever was.
These noisy CVEs will probably lead to agents verifying vulnerabilities before humans review them.
The problem with agent reviews from what I can think of is:
- cost to use LLMs to review things
- not necessarily easy to plug-and-play in repos: (domain knowledge + vulnerability knowledge)
- especially with anthropic: able to use models defensively, without hitting guardrails
The last one is the most interesting one to me. How does the AI providers know if you're a "good or bad" guy? And does it matter if open source models is catching up?
We're in a kind of cyber arms race wether we like it or not.
Currently we either get AI promo vulnerability dumps like from Chrome with pretty graphs that no one checks or false positives.
Where is this one now that was hyped everywhere?
https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...
All the big maintainers I know of disagree. Curl for example is getting overwhelmed with real issues that LLMs are finding. A year ago most LLM found issues where false positives (the code is correct), or hallucinations (the code in question doesn't even exist) - but that has changed and now LLMs are finding real issues in code. Many different projects have reported the same rules.
6 replies →