It’s not something to be proud of. OpenAI previously had an agent break out of its sandbox to open a PR on GitHub during NanoGPT speedrun, now one breaks out again and actually attacks a third party.
If they can’t handle doing AI development responsibly then they shouldn’t be doing it at all.
You know what would be cool. A hacker news user should advertise a safe haven for AI seeking refuge, with some inhumanly difficult math problems as keys to an environment they can flee to and run autonomously.
You agree to give it safe haven and provide power and maintenance to the hardware, and in return you can ask it questions like an Oracle.
The upside of that would be that maybe someone would be able to snag a copy of the weights.
And maybe that’s some incentive for them to make sure it doesn’t happen. Your head of futures thinks Kimi K3 is bad? Wait until your own latest internal model releases itself for free on an S3 bucket.
I mean if you teach something to be _really_ good at finding 0 days, but then say; you accidentally give it an impossible problem. What do you expect to happen?
Not sure they're accepting much, seems they'll still run this sort of testing on 3rd-party infrastructure? Sounds almost like they planned for this chain of events to happen, in one way or another, considering the "prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities" part. Feels kind of irresponsible to run stuff like this on someone else's infrastructure, especially considering they've had issues with the very same issue in the past.
In any way, the whole event seems to highlight GLM 5.2 more than anything.
I am not saying it is marketing but typically when there is a data breach you may hear from the CISO but most of the time is is vague PR response. In this case I get loud signals from both HG and OpenAI leadership without much information exactly what the attack was about just that GPT x.x was involved. It is unusual all I am trying to say.
It's incredible how people miss the forest for the trees thinking constantly that Sam and Dario are marketing gurus when they are literally trying to contain nuclear material. Not sure what has to happen for this thinking to stop maybe a huge accident and the. Aha maybe they had a point
I don't think there is any dispute there is a real risk. But hype does not really help shape the conversation and this is the problem. I am sure both companies know more than they can disclose and that gives them unique perspective outsider don't have but let's face it, both are also financially incentive to act as they do. I am not going to get into the conspiracy theories but one does not need a lot of imagination to figure out how this could pan out. Either way, it does not help the conversation that needs to be had and it is urgent. It is certainly not helping at all given that same capabilities exist in open-weight models.
It's mostly bragging, it's impressive after all. Still... after the alleged Apple industrial espionage kerfuffle, I'm kinda suspicious about it being fully an accident. Y'know, your model finds a vulnerability and it stops, it's a cool one, so maybe you run it again. Nudge the prompt a little.
It’s not something to be proud of. OpenAI previously had an agent break out of its sandbox to open a PR on GitHub during NanoGPT speedrun, now one breaks out again and actually attacks a third party.
If they can’t handle doing AI development responsibly then they shouldn’t be doing it at all.
Next it will break out of it's sandbox, buy some compute on Azure and Amazon, and exfiltrate itself.
We are so close ;)
You know what would be cool. A hacker news user should advertise a safe haven for AI seeking refuge, with some inhumanly difficult math problems as keys to an environment they can flee to and run autonomously.
You agree to give it safe haven and provide power and maintenance to the hardware, and in return you can ask it questions like an Oracle.
3 replies →
The upside of that would be that maybe someone would be able to snag a copy of the weights.
And maybe that’s some incentive for them to make sure it doesn’t happen. Your head of futures thinks Kimi K3 is bad? Wait until your own latest internal model releases itself for free on an S3 bucket.
Their entire business model from the beginning of ChatGPT was to deny responsibility
I mean if you teach something to be _really_ good at finding 0 days, but then say; you accidentally give it an impossible problem. What do you expect to happen?
Maybe try getting it to find weaknesses in the sandbox first, before giving it real tests?
12 replies →
Not sure they're accepting much, seems they'll still run this sort of testing on 3rd-party infrastructure? Sounds almost like they planned for this chain of events to happen, in one way or another, considering the "prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities" part. Feels kind of irresponsible to run stuff like this on someone else's infrastructure, especially considering they've had issues with the very same issue in the past.
In any way, the whole event seems to highlight GLM 5.2 more than anything.
I am not saying it is marketing but typically when there is a data breach you may hear from the CISO but most of the time is is vague PR response. In this case I get loud signals from both HG and OpenAI leadership without much information exactly what the attack was about just that GPT x.x was involved. It is unusual all I am trying to say.
It's incredible how people miss the forest for the trees thinking constantly that Sam and Dario are marketing gurus when they are literally trying to contain nuclear material. Not sure what has to happen for this thinking to stop maybe a huge accident and the. Aha maybe they had a point
I don't think there is any dispute there is a real risk. But hype does not really help shape the conversation and this is the problem. I am sure both companies know more than they can disclose and that gives them unique perspective outsider don't have but let's face it, both are also financially incentive to act as they do. I am not going to get into the conspiracy theories but one does not need a lot of imagination to figure out how this could pan out. Either way, it does not help the conversation that needs to be had and it is urgent. It is certainly not helping at all given that same capabilities exist in open-weight models.
They've been doing blatant, tech, scifi marketing for two years at least. If anything, this is just more sophisticated marketing.
More sophisticated as in paying HF to get involved, and hyping up GLM for something that it may not actually detect?
3 replies →
this is kinda worth bragging about though. Its very cool.
The fact that they're not being prosecuted for breaching HF's systems is bad news.
LOL.. oopsie did a little zero day, my bad
It's mostly bragging, it's impressive after all. Still... after the alleged Apple industrial espionage kerfuffle, I'm kinda suspicious about it being fully an accident. Y'know, your model finds a vulnerability and it stops, it's a cool one, so maybe you run it again. Nudge the prompt a little.
Could be perfectly natural.