Comment by andy99
1 month ago
> If "Mythos-class" models are such a problem, then... why not just let it fix everyone's code?
Because it doesn’t really confer the advantage they claim, especially compared to e.g. paying an equivalent amount of money to do traditional security scanning.
It’s much better to play of FOMO and hype than to let everyone use it and be underwhelmed.
Are you claiming that LLMs aren't finding new issues compared to previous methods?
There's a huge number of security issues coming out in recent months, especially via Anthropic (glasswing etc). We don't have to take their word for it: look at the code. Some open source maintainers are talking about burnout due to spending so much time patching.
They're probably creating way more vulnerabilities than they're solving. Go look at OpenCode and tell me that any of that is sane. Or fuck, the OG of vibe coding, Claude basically is a terrifying attack vector. It's poorly reviewed and open to prompt injection and yet it basically has access to whatever the user has access to on most corporate machines. They can't even fix flickering bugs but somehow we're supposed to trust that they aren't opening our machines up to terrifying vulnerabilities? It's amazing to me that people will just let it run arbitrarily bash commands in their home directory without thinking, but all the sudden act gravely concerned about security in the age of overhyped LLMs.
I do think security issues in core building blocks like curl and the linux kernel (and almost every significant project) are still a concern even if developers are being sloppy on newly-built apps.
This isn't an "are LLMs net good or bad" argument. It's "are they finding many new security issues or not?". If it's the latter, we want to deal with it no matter where the issues are coming from.
(see: https://news.ycombinator.com/item?id=49077452 )
The suggestion was to have some AI system “fix” the code. They are reporting that they’ve found lots of bugs. Are they even claiming to have exhaustively found all the bugs? I don’t think even the most optimistic pitches would claim that.
I’d expect patching existing codebases to be an eternal treadmill as better models come about.
Who's suggesting that? They're sending reports to projects to fix. It's up to the projects on how they fix them.
No, I don't think all bugs are fixed. The point of the project (glasswing etc) was to fix as many as possible in the core software the world runs on before the capability to find vulnerabilities is available to everyone (black hats included). Which may only be a few months.
I do think everyone expects it to be an ongoing treadmill: models get better, find better vulnerabilities, etc.
Yeah, there’s a lot of people that are in the “AI doesn’t work” camp. IDK what to tell them except that they are holding it wrong. My Anthropic subscription (in the hands of an experienced developer) is worth 4 mid tier or 2 top tier devs. And makes better code than the mids. If you “hold it right”.
I think you're describing a strawman. As a proper hater, I know these things have some useful functionality, but most of us don't think "stochastic tool that can do some useful things but also frequently fucks up" is worth two trillion dollars and massive overhyping from the most irritating people on the planet who can't even tell good code from bad.
3 replies →
Spending their time patching, or reviewing slop "patches"?
They're not slop, if you're talking about recent ones. You might still be operating on information for a year or two ago.
Here's the curl project talking about the strain they're under from real reports (despite being a mature and well-vetted project):
> A thirty years old project could make you think you’ve seen most things already, but we have not been in this situation before.
> The rate of incoming security reports is 4-5 times higher than it was in 2024 and double the speed of 2025 – meaning that on average we now get more than one report per day. The quality is way higher than ever before. The reports are typically very detailed and long.
- https://daniel.haxx.se/blog/2026/05/26/the-pressure/
---
Linux kernel maintainer Greg Kroah-Hartman:
> "Something happened a month ago, and the world switched. Now we have real reports." It's not just Linux, he continued. "All open source projects have real reports that are made with AI, but they're good, and they're real." Security teams across major open source projects talk informally and frequently, he noted, and everyone is seeing the same shift. "All open source security teams are hitting this right now."
- https://www.theregister.com/software/2026/03/26/linux-kernel...
---
And ffmpeg, who previously complained about slop, 2025: https://xcancel.com/FFmpeg/status/1984220199193891166
Now say serious issues are being found, 2026: https://xcancel.com/FFmpeg/status/2066169070387413147
(I only point out their previous stance to show that they're not coming from pure AI hype.)
3 replies →