Comment by stuxnet79
6 hours ago
We have a ton of new vulnerabilities that have emerged from widespread use of AI systems. But at the same time the frontier AI labs are releasing tools like Mythos that purport to automate/simplify the identification of vulnerabilities.
Between these two trends, I struggle to see what the future holds for the security industry.
Either way, as is always the case with the tech industry, the incumbents in this space will be getting paid the big bucks and the consumer will ultimately hold the bag. We absolutely need tougher data privacy / security laws & I wonder what catastrophic event will force law makers and voters to take this issue seriously.
My feeling is the defender wins in the long-run. There's only a finite number of bugs and vulnerabilities.
Semi agreed but I think that we are likely to see a ton of vulnerabilities found in the near term as AI's go through codebases and find all the stuff that was missed over the years. Once that period has (mostly) passed I imagine things will slowdown to somewhat similar to a normal stream of bugs and vulns and as new code is created.
Surely there is a mathematical model here, but intuitively I do think there is an infinite number of typos and errors you could contain in the set of finite books, and similarly there would be an unlimited number of bugs and vulns in the set of Turing machines.
> There's only a finite number of bugs and vulnerabilities.
The context of an LLM is also finite.
Vulnerabilities are perpetually being created, and this will be true no matter how good LLMs become at writing code - there's simply too many factors that can contribute to something apparently benign becoming dangerous.
Lots of bugs seem to be fundamentally quite local, but potentially with global trigger conditions. Heart bleed for example could've been avoided even if you could only read small segments of the codebase at a time, but could only be triggered with more context.
I suspect that a combination of ai and memory safe languages will really shine in the next decade.
I doubt you can prove that.
Do you think the attacker or defender will have been the overall beneficiary of LLMs when we look back in 5 years from now?
1 reply →