Comment by rishabhaiover
16 hours ago
What is happening? I see multiple outages and CVEs is being reported on HN's front page. I've never seen these many security/incident related posts on HN's front page.
16 hours ago
What is happening? I see multiple outages and CVEs is being reported on HN's front page. I've never seen these many security/incident related posts on HN's front page.
Some combination of reporting bias given concerns about LLM security capabilities and actual new vulnerabilities found with LLM assistance. Even if exploits and outages are unrelated to LLMs, I'm certainly thinking about whether claude could build these things (or if actors already have).
> What is happening?
Slowly at first, and then suddenly. AI assisted anything follows this trend. As capabilities improve, new avenues become "good enough" to automate. Today is security.
i believe a good portion of the cves hitting the front page are moreso because they are ai-related (found partially/in whole by ai) and make for quick upvotes.
AI is happening.
In each recent case?
AI assistance was explicitly disclosed on yesterday's. Today's has Claude as one of two contributors on this GitHub Pages site at least so it's also very likely.
Agents are capable of finding this kind of stuff now and people are having a field day using them to find high-profile CVEs for fun or profit.
In some sense, I wonder if non-open-source is "safer" since LLMs can't mass scan the code for exploits.
Maybe for a while, but there's nothing stopping LLMs from examining disassembler output.
Security through obscurity
If they don't get scanned, then they also don't get fixed, so if they have the same amount of holes, they will stay vulnerable for longer.
It's actually the perfect evergreen content to discuss on HN in an age where so much else is AI generated.
Automated vulnerability discovery via LLM.
Anyone care to share which models and which prompts actually lead to finding these kinds of vulnerabilities? Or the narrowing-down workflow that can get an LLM to discover them? Surely just telling claude "Find all vulnerabilities in this project LOL" isn't enough? I hope?
The Anthropic researchers have said their flow is as simple as:
1. Pick a file to seed as a starting place.
2. Ask the LLM (in an agent harness) to find a vulnerability by starting there.
3. If it claims to have found something, ask another one to create an exploit/verify it/prove it or whatever.
4. If both conclude there is a vuln, then with the latest models you almost certainly found something real.
Just run it against every file in a repo, or select a subset, or have an LLM select files with a simple "what X files look likely to have vulns?".
So basically yes, it is that simple. It's just a matter of having the money to pay for the tokens.
1 reply →
[dead]
Everyone was talking about how Mythos was overblown marketing, and while it may be, they missed the forest for the trees. Capabilities have been escalating for a year now and we're at the point of widespread impact. I don't suspect we'll see a slowdown for a long time.
I agree. It is not like Mythos or other LLMs are insanely smart/superhuman. Many of these vulnerabilities could be discovered fairly easily by trained human experts as well. The problem is more that it requires an insane amount of attention and time of highly-paid experts to shake out these issues vs. an LLM that never gets tired and can analyze a large amount of code at low cost.
Linus' law was wrong because there were never enough (qualified) eyeballs to check the code. LLMs provide an ample supply of eyeballs (though it's not a benefit to open source, since proprietary developers can use the same LLMs).
Same applies to them being good enough to program, but many are so focused on source code generation that they don't get the whole picture.
Thanks to agents and tool calling, there are now business cases that can be fully described by AI tooling, the next step in microservices, serverless and what not.
Naturally with a much smaller team than what was required previously.
A mix of AI and hybrid warfare.
Perhaps it was the prior quiescent period that was the anomaly.
I wonder where are the Rust naysayers hiding now
C code is broken - period
... there's also a bit of a frequency illusion factor.