Comment by miroand1
2 days ago
We are in the endgame now it seems.
Hard to see take-off stopping or slowing down. China open-source basically guarantees it.
"May you live in interesting times" - as they say.
2 days ago
We are in the endgame now it seems.
Hard to see take-off stopping or slowing down. China open-source basically guarantees it.
"May you live in interesting times" - as they say.
> Hard to see take-off stopping or slowing down.
It's hard to see takeoff at all. This was a long-horizon adversarial task burning millions of tokens. It rolled a mediocre, detectable exploit chain, and now OpenAI is proud of it.
Case in point, GLM-5.2 has been weights-available for several weeks now. No life-changing cyber attacks have transpired, no novel chemical/biological/nuclear weapons were made in some guy's backyard.
1. it's not cheap to run glm-5.2 so not just anyone can do it 2. just because you haven't heard of attacks doesn't mean they haven't happened 3. this attack in the article was performed by a prerelease model which presumably benchmarks a bit above Sol which benchmarks above glm-5.2
We went from gpt 3 to models discovering and chaining their own zero days in a couple years. I'm not sure what else "takeoff" could possibly look like?
GLM has an extremely cheap subscription plan similar to Claude Code from Z.ai. You get Opus-level quotas with 5.2 and none of the Anthropic-style model nerfs when you ask cybersecurity questions. It's extraordinarily, preeminently accessible to anyone that wants to use it for ill or good.
> We went from gpt 3 to models discovering and chaining their own zero days in a couple years. I'm not sure what else "takeoff" could possibly look like?
GPT-3 can discover and chain their own zero days too, if the targeted software is vulnerable to enough low-hanging fruit. Exploit chains are not a reflection of intelligence, but more often a reflection of architectural oversights that can be tested with common exploits like XSS or bruteforcing.
2 replies →
> This was a long-horizon, unsupervised task burning millions of tokens.
As if the immediate future wasn't billions of these tasks... Many successfully improving their own capabilities
> As if the immediate future wasn't billions of these tasks...
There's only so many GPUs and a lot of them are devoted to patching flaws.
> Many successfully improving their own capabilities
I haven't seen much of that. But that also applies to the ones on defense.
And more flaws are probably going to take increasing resources to find.
6 replies →
Did you ignore the number of new exploits in the last month?
Big financial institutions are panicked at the new attacks and how easy it is to poke holes in their systems.
> Big financial institutions are panicked at the new attacks and how easy it is to poke holes in their systems.
Have any big financial institutions been hacked with an AI-generated payload, then?
I've been following the number of new exploits; it's not really any higher than it was 12 months ago.
1 reply →
> Hard to see take-off stopping
I think it's reasonable to assume that we're close to, or already at superhuman cybersecurity capabilities at certain domains. But reaching superhuman abilities at one domain doesn't guarantee proficiency at others. Our world would still change if all the models could do was to find exploits in software, but this doesn't guarantee any type of 'take off' towards other domains, therefore I wouldn't phrase it as one.
Models are already being used to defraud people, now that's being driven by other people at the moment but doesnt seem that difficult of jump. Giving themselves a way to make money will be a pretty big jump.