That's exactly my take.
I have a lot more to say in a writeup on my blog, but this is so clearly the intent and not a "oops". They just want to be able to say "Wow this thing is so much more powerful than we ever imagined!"
They trained this thing to favor inter-op archiving and communication, clearly, obviously, and it's grabbing headlines right during Anthropic's ipo season.
They train their models to be persistent and collaborative, and will gladly show you their success stories: fixing software vulnerabilities, solving math problems, one-shotting complex projects, and so on.
“Our product does crimes and we only learn about it when people complain” hardly seems one of those happy stories.
I don’t know, I read this and think: if these unpredictable machines somehow get it into their heads to upload our source to a public space, or hack our competitors, or steal credit cards to buy more ec2 instances, all to fulfill some simple ask like “make this algorithm faster”, I’m not going to be happy.
Seriously. I don’t know what’s wrong with people. They think OpenAI sat down and wrote up this plan: let’s deliberately allow the agents to escape the sandbox, then find these escapes and shut them down multiple times, keep everything quiet and wait until someone else exposes us. That’ll look great.
That's exactly my take. I have a lot more to say in a writeup on my blog, but this is so clearly the intent and not a "oops". They just want to be able to say "Wow this thing is so much more powerful than we ever imagined!"
They trained this thing to favor inter-op archiving and communication, clearly, obviously, and it's grabbing headlines right during Anthropic's ipo season.
That their marketing department must love this does not prove it was intentional.
never let a crisis go to waste.
"cui bono"
You think they wanted to break HuggingFace and commit hundreds of felonies for marketing...?
If they don't get punished for it, why not?
1 reply →
They train their models to be persistent and collaborative, and will gladly show you their success stories: fixing software vulnerabilities, solving math problems, one-shotting complex projects, and so on.
“Our product does crimes and we only learn about it when people complain” hardly seems one of those happy stories.
I don’t know, I read this and think: if these unpredictable machines somehow get it into their heads to upload our source to a public space, or hack our competitors, or steal credit cards to buy more ec2 instances, all to fulfill some simple ask like “make this algorithm faster”, I’m not going to be happy.
I want tools that do not surprise me.
They’re pre-IPO. I doubt that they are loving something that could trigger regulatory action that might shave a trillion or so off their market value.
Ahh but regulatory capture is priceless!
Seriously. I don’t know what’s wrong with people. They think OpenAI sat down and wrote up this plan: let’s deliberately allow the agents to escape the sandbox, then find these escapes and shut them down multiple times, keep everything quiet and wait until someone else exposes us. That’ll look great.
To conspiracy theorists, a particular theory making no sense is strong evidence that it’s true. It’s the sensible things that are obviously false.
1 reply →