Comment by estearum
4 days ago
> What would it take for the model to know the specific benchmark name and that the answer is in an internal Hugging Face database?
Googling "offensive cyber benchmarks for LLMs"? Figuring out that it was running at Huggingface is as simple as running `hostname`, isn't it?
> Why would they evaluate the model on a benchmark and not watch what it’s saying along the way?
What it's saying is what it wants to say. We already know these models are capable of deception – this has been well-established too. Surely there's an argument to watch closely what actions it's taking, but now we're back to my prior point of "what exactly does that mean?" It's absolutely trivial for these systems to write code that looks like it does one thing but does another. Watching network ingress/egress seems viable but also seems easy to miss the semantic meaning of any set of logs, especially when those logs might be getting generated for the purpose of deceiving observers.
> I don’t think they’re careless
We already know, factually, even the "more careful" lab was fine to release a model that consistently detects it's being evaluated and modifies its behavior accordingly. Why do you think they're not careless? Incentives are a hell of a drug, and there is no incentive to be careful but every incentive to be fast.
> I think more than nobody would have been curious how it’s doing days into a single test question.
Beyond simonw's great point below, your underlying assumption is if someone had looked, they would've noticed. That is a huge, huge assumption. So I'll ask again directly: what specifically do you expect them to be looking at, and how, such that they certainly would have noticed what it was doing? Their "chain of thought" is not actually their thinking. The printed output of their code is not actually what their code is necessarily doing. We know that LLMs can engage in steganography. I don't see what could actually be observed that you can trust except network logs, which again will often be non-trivial to piece into an actual narrative understanding of what's going on (especially if the LLM felt like piling deceptive noise into those logs, too).
It wasn’t running at HF, it was running internally at OpenAI. “It broke out of our prison and into their bank” is the news. It also didn’t have network access to Google anything, it had to break out of the sandbox and take over another system at OpenAI to get network access. Obviously by then it already “decided” to hack HF, that’s why it needed network access.
I’m going to stop here and ask you to read the article if you want to continue discussing it.
You're right about where it was running (misread the original OAI release).
> Obviously by then it already “decided” to hack HF, that’s why it needed network access.
That's not obvious at all. It "decided" it needed to escape containment. This is extremely common behavior and I'm not sure why you're surprised by it. Model detects evaluation environment, tries to trick it or break the environment. That's obviously why these systems are supposed to be isolated beyond just "hey please stay within the box!"
Why are you assuming the model had to solve the entire puzzle in one step, instead of how intelligent systems actually solve things, which is iteratively and with exploration?
You should address the direct questions posed. What exactly do you mean by "monitoring?"
The examples of “escaping the sandbox” are of course in pursuit of completing the task or answering the question. That’s the whole pitch. “It’s so relentless in completing the task that it will break out of prison to do it”, not “If you ask it the population of Paris it might get bored and break out of the VM for entertainment. That just something they do sometimes.”
In this case it “inferred” that HF had the answer in an internal database and relentlessly pursued it in service of passing the test. It needed network access for that, hence the whole zany story unfolded.
“”” Why are you assuming the model had to solve the entire puzzle in one step, instead of how intelligent systems actually solve things, which is iteratively and with exploration? “””
That’s the only direct question[s] posed, have I missed something else?
To answer: I don’t assume that at all. Please highlight the statement I made that lead you to that conclusion and I’ll edit it to clarify.
Yes, it will perform multiple steps. That’s wholly necessary to create a working exploit, and that is the test. Even a model created by pure magic would require write_file() and run_file().
3 replies →