Comment by simonw
4 days ago
I think your head is in the sand if you prefer to believe that this was a hoax for marketing purposes as opposed to accepting how effective these models have become at exploit research.
You're welcome to think they are exploring what happened for marketing if you like. I didn't get that tone from their post about it myself but I don't hold a particularly strong opinion on that.
I didn't say it's a hoax. AI's exploiting common, unpatched, or preventable vulnerabilities just isn't far above what script kiddies and network scanners have done for decades. I literally pitched it as a business model to some people before Windows XP was invented. Ultimately, I decided against what I felt was unethical but would've scored easy money with scary, automated reports.
How these events are described in most articles uses wording that makes people feel the whole situation has changed and you might want to buy these products due to their scary reports. If they said what I said, or what tptacek said, many people wouldn't care about it much more than non-AI, security products or pentesting services they've been buying (or ignoring) for 10-20 years.
Also, you shouldn't interpret posts in isolation: we must consider patterns of behavior (character). They've consistently overhyped what AI's do and what value it provides to businesses and how we're all going to be unemployed/dead. They've done this to increase sales or market value pre-IPO. Then, some of them publish another set of articles promoting a specific, AI tool in a similar way.
So, the proper interpretation is to see this as the kind of talk they're always doing for marketing. The AI sellers are creatures of habit. We should highly-skeptically and scientifically evaluate every model. We should also compare them to existing, security practices. For instance, would the attack have happened with memory-safe systems, proper hardening, and network/web apps with built-in security?
Do we need an AI? Or should we use techniques like Burroughs B5000's memory safety (1961) or secure, distributed libraries? Will OpenAI and HuggingFace tell you to spend more money on the latter to their AI's can't hack your systems without inventing RAM-based attacks or something? Probably not because these are marketing pieces, not security advice.
Exploiting, not exploring.