Comment by pizza234

1 hour ago

> Like another comment already pointed out, that sandbox OpenAI used was the equivalent of a wet paper bag.

I wouldn't be sure about even well-configured jails to be safe from agents. AIs escaping jail using zero-days are happening, just search. I'm not saying that they're useless, but that will be still very risky.

Actually doing the same incident, agents did escape the sandboxes (doc here: https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78...):

> On July 9, an internal-only research agent tasked with completing an ExploitGym evaluation was able to obtain root access within the parent virtual machine of the testing sandbox. Later that night, a second internal-only research agent independently obtained the same access. That second agent then attempted to reach the internet, but was denied by OpenAI’s outbound network controls. It subsequently bypassed those controls by exploiting certain Artifactory endpoints, gaining access to the public internet.

although OpenAI is light on details.

> The model merely requests that your harness do something.

I think that a control/execution layer absolutely makes sense and it's part of defense in depth, but in itself is not a leash: for example, a nefarious action can be composed by legitimate actions, or it can be obfuscated. Or, interesting case: an action itself that is formally legitimate, but has nefarious side effects (e.g. a GET that with certain parameters crashes a server).

Somewhat related, but on a general line, a very common argument against AI apocalypse is that it requires access to the physical world (which in a way, is a jail). Well, as Hinton said in an interview, imagine AI as a rich person at home with internet; they don't need to do physical attacks themselves - they just pay real people to do that (tee hee https://rentahuman.ai). But even without money, another scenario from Hinton: if a sufficiently advanced AI wants to assault the Capitol, they don't need physical access - they just persuade a group of people to do it.