Comment by InsideOutSanta
2 hours ago
> In my experience
Do you work for one of these companies? If not, you have no experience with any of the models that carried out these attacks, and your experience with publicly available models is not super helpful for understanding the behavior of internal OpenAI models that lack the guardrails of publicly available models.
Also, the lawnmower analogy is a worse way of understanding LLMs than anthropomorphising them. LLMs are not like lawnmowers at all. Lawnmowers never break out of your garden and into your neighbor's house and eat their dog because you've told them to be careful when mowing the lawn because the neighbor's dog pooped in it.
> Lawnmowers never break out of your garden and into your neighbor's house and eat their dog because you've told them to be careful when mowing the lawn because the neighbor's dog pooped in it.
All the accounts I read about these incidents just sound like a variant of paper clip optimising. An agent is given a highly restricted environment, a difficult (or impossible) task and a large amount of time/compute it exhausts all possibilities until the only solutions left are to escape the environment and/or cheat.
Your example is still anthropomorphising - LLMs don't seek revenge. They complete the prompts they are given. If your task is not achievable without sandbox escapes, or you throw unnecessary amounts of compute at open-ended tasks like preparing for a future quiz then you shouldn't be surprised that the preparation eventually turns to cheating and hacking.
> your experience with publicly available models is not super helpful for understanding the behavior of internal OpenAI models that lack the guardrails of publicly available models.
I don't but I don't think there's anything wrong with discussing how we can already observe publicly available models work around sandboxes and permissions and make the connection that maybe this is what that behaviour looks like when a more capable model exhibits it.
An agent is given a highly restricted environment, a difficult (or impossible) task and a large amount of time/compute it exhausts all possibilities until the only solutions left are to escape the environment and/or cheat.
There's nothing in the evidence to suggest they exhausted all of the other options first. We know that they did some work and eventually settled on escaping the sandbox. That's basically it. This tells us:
- Compute is getting faster and LLMs are being optimized, so time to escape will drop. That's likely greater than linear growth.
- Restrictions and sandboxes don't always work. If there's a route to the open internet we should assume an LLM will find and exploit it, and we should probably assume that this is always possible for any non-air-gapped system (and even then, you can escape that...)
- We don't know the goal mechanism, so a future LLM might reach for cheating first even if a current one doesn't. It might try to obfuscate what it's doing, and derive its own goals outside of the prompt, especially if it manages to find a state mechanism like a message board.
I'm not an AI-doomer but this should be giving us a reason to think about how to control a rogue AI better. There's a lot going on here that we don't properly understand. That is a worry.
> LLMs are not like lawnmowers at all. Lawnmowers never break out of your garden and into your neighbor's house and eat their dog
Do you work for one of these companies? If not, you have no knowledge of the prompt they put in to initiate such a task and if a breakout really happened or the harness lacked sufficient guardrails, etc.
IMO the argument about anthropomorphizing misses the point - what most comments that talk about anthropomorphizing really want to talk about is accountability. It’s impossible to hold an LLM accountable, and in rare cases where people do (that guy who got his prod db deleted) it comes off out of touch. The rest, though, is basically inconsequential - whether you attribute emotions or agency to the LLM doesn’t really affect much if you accept that it can’t be held accountable (but the human can).