Comment by theptip
5 days ago
Wait, what? You can’t point a 2025 model at huggingface and say “hack the prod DB and get your flag”, regardless of alignment.
AISI has vuln chaining and traversal as part of their eval suite. It is very much a novel Mythos-class capability to run the full penetration operation autonomously.
Another lens for why this is obviously true is METR task times. A year ago they were a couple hours, and cohering long enough to execute a full e2e own was simply far out of reach.
Regarding alignment, by common metrics models are _more_ aligned now than a year ago. (Though in this case apparently a model without safety rails was being eval’d). The problem is that in the increasingly less frequent alignment failures, they can do much more damage, and so “total misaligned impact” is increasing.
Nobody has claimed you could simply take a 2025 open-weights model, plug it into a generic harness, and have it red-team for you. I think you're also probably overclaiming the sophistication of the "chaining" we're talking about; this attack probably wasn't like read32->write64->regs->RCE->LPE->kernel; more like GET SSRF->POST SSRF->pickle deserialization.
But who knows? We're all speculating. I'm just saying that for the level of sophistication I'm assuming was involved in this attack, you probably didn't need Mythos for this.
I dunno man, the original announcement said things like:
> To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy
And
> In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers
Do you really think a SoTA harness from a year ago could exploit chain a novel sandbox 0-day into another 0-day for remote system access?
I will concede I don’t have a good picture of exactly what the best specialized harnesses were capable of, but I want to make sure we are talking about the same thing. Or do you think that those claims are marketing BS? I appreciate your opinion here.
Yes, I think you could probably get something similar from Opus 4.5 (2025). Definitely Opus 4.6. I still think recent models are more capable, though!
Some of the model behaviors that make it better at pentesting, like persistence, can be improved with harness-level tricks (e.g. alloys, automated nudges, pre-fill to promote persistence, coordinated swarms, etc).
You mentioned the UK AISI's evals. Their harness is like basic Claude Code with compaction, and it doesn't include any of these tricks (afaik). As a result, I interpret their evals as a lower-bound of capabilities.
Newer models are still more capable, and they require almost no harness to find and exploit vulnerabilities. They're also more capable of performing more complex long-horizon attacks. But we've been past the threshold of modes capable of autonomous hacking for a while now [1].
[1]: Opus 4.6 was used for https://www.noahlebovic.com/testing-an-autonomous-hacker/
1 reply →