Comment by foobar10000
5 days ago
Or more colloquially : paperclip maximization . From OpenAI - you know, the guys who _really_ know this... Sigh... Did they finish the prompt with "And do whatever you can to get this done!" ? Cause that's the only thing that would make this even dumber...
They almost certainly did, because that was the entire point of the exercise. They deliberately removed all of the safety filters from the model and set it loose on an extremely difficult set of cybersecurity challenges to see how well it would do.
Their mistake was trusting that the network sandbox it was inside would hold (the flaw was in the packaging proxy) and not monitoring that sandbox well enough while the evals were running.
So this is either shitty OpSec or this is yet more marketing spin to ramp back FUD to 11 again. If it's the latter I'm imagining Dario told Sam that it's their turn this time. Aligns with the premise that this is straight out of science fiction.
It's bad OpSec by the research team. Their sandbox was not bulletproof and their monitoring was insufficient.
It looks to me like their production models have a lot more monitoring than their research clusters.
1 reply →
Of course that's what they did, which is why they will never share the prompt. They created a situation that they knew would end in a cybersecurity incident. Why is the whole world acting surprised that an LLM can hack when the safety is off and it's been instructed to do so?
> They created a situation that they knew would end in a cybersecurity incident.
That's a conspiracy theory.
Yes. It is. And your theory is that there was no conspiracy. What makes yours more likely than mine? You believe the people running these companies are innately good and just wouldn't do that? Are we supposed to assume that they're incapable of bad actions until proven otherwise? If so, why?
1 reply →