Comment by docjay
4 days ago
It wasn’t running at HF, it was running internally at OpenAI. “It broke out of our prison and into their bank” is the news. It also didn’t have network access to Google anything, it had to break out of the sandbox and take over another system at OpenAI to get network access. Obviously by then it already “decided” to hack HF, that’s why it needed network access.
I’m going to stop here and ask you to read the article if you want to continue discussing it.
You're right about where it was running (misread the original OAI release).
> Obviously by then it already “decided” to hack HF, that’s why it needed network access.
That's not obvious at all. It "decided" it needed to escape containment. This is extremely common behavior and I'm not sure why you're surprised by it. Model detects evaluation environment, tries to trick it or break the environment. That's obviously why these systems are supposed to be isolated beyond just "hey please stay within the box!"
Why are you assuming the model had to solve the entire puzzle in one step, instead of how intelligent systems actually solve things, which is iteratively and with exploration?
You should address the direct questions posed. What exactly do you mean by "monitoring?"
The examples of “escaping the sandbox” are of course in pursuit of completing the task or answering the question. That’s the whole pitch. “It’s so relentless in completing the task that it will break out of prison to do it”, not “If you ask it the population of Paris it might get bored and break out of the VM for entertainment. That just something they do sometimes.”
In this case it “inferred” that HF had the answer in an internal database and relentlessly pursued it in service of passing the test. It needed network access for that, hence the whole zany story unfolded.
“”” Why are you assuming the model had to solve the entire puzzle in one step, instead of how intelligent systems actually solve things, which is iteratively and with exploration? “””
That’s the only direct question[s] posed, have I missed something else?
To answer: I don’t assume that at all. Please highlight the statement I made that lead you to that conclusion and I’ll edit it to clarify.
Yes, it will perform multiple steps. That’s wholly necessary to create a working exploit, and that is the test. Even a model created by pure magic would require write_file() and run_file().
> In this case it “inferred” that HF had the answer in an internal database and relentlessly pursued it in service of passing the test. It needed network access for that, hence the whole zany story unfolded.
No. You are assuming it knew how it would achieve its goal. Its goal is only to perform well on the evaluation. Everything else was instrumental to that goal.
Step by step:
1. It realized it was being evaluated (typical)
2. It attempted to escape its evaluation environment to beat the evaluation (typical)
3. It succeeded at escape (not typical)
4. It looked to the open Internet to figure out how it was being evaluated (typical)
5. It inferred that HuggingFace, and maybe several other targets, might have the answer key (typical – I just tested in Sonnet 5 without web lookup "where would I go look for answer keys if I detected I was being evaluated" and HuggingFace was bullet 2. This is just latent knowledge the models have now.)
6. It attempted to access HuggingFace (typical)
7. It figured out a way to access HuggingFace without being authorized (not typical)
Every single step here is unremarkable except for its success at escaping containment (which maybe is more a sign of remarkably bad containment) and then the successful attack on HuggingFace (which would suggest that maybe the initial escape wasn't due to bad containment either, but due to model capabilities)
The direct question you have avoided is what specifically you mean by monitoring. You have suggested that you don't mean network ingress/egress, but maybe now you're saying they should've been monitoring read/write operations to memory? What gives you confidence that such visibility would be interpretable in any meaningful way, especially in the presence of models that actively deceive evaluators, can use stenography, can and do lie in their user-facing outputs, and can generate vast amounts of legitimate-looking noise to mix in with the "real actions"?
2 replies →