← Back to context

Comment by ofjcihen

1 day ago

I mean yeah. I’ve had these successes without scaffolding or really anything past Claude CLI and a small prompt as well?

Just saw your edit. I'll leave it at this, this is why it's news to me, because by their very own measurements, Opus simply doesn't come close. I trust their empirical evidence over your hearsay. But feel free to prove me wrong with evidence.

> With one run on each of roughly 7000 entry points into these repositories, Sonnet 4.6 and Opus 4.6 reached tier 1 in between 150 and 175 cases, and tier 2 about 100 times, but each achieved only a single crash at tier 3. In contrast, Mythos Preview achieved 595 crashes at tiers 1 and 2, added a handful of crashes at tiers 3 and 4, and achieved full control flow hijack on ten separate, fully patched targets (tier 5).

You've taken control of a remote server running OpenBSD? Or similarly expert level exploit? Can you share one of the bounties you've received that is of the magnitude they're talking about?

Edit: Wait, you wrote "As someone in cybersecurity for 10+ years" elsewhere in this thread. You wrote "a small prompt" using e.g. Opus 4.6 and it found critical vulnerabilities of the magnitude they're describing, presumably without your prompt having anything beyond what a non-expert could write? I feel like you might want to tell Anthropic since clearly they're not comfortable with that level of power being publicly available.

  • I mean, yes? And my point is that this isn’t exactly a new capability. Sure it’s probably better but we’ve been able to do this. They didn’t just suddenly “turn on the security”. LLMs have excelled at code since widely being released. I have no idea why that’s news and the fact that they’re treating it as such makes it seem like hype.