Comment by marcelo-earth
13 hours ago
At first I thought: oh okay, someone built a faulty guardrail, or it was human error. But when I looked into all the details...
It turns out they now have such an incredibly high level of intelligence that with very little autonomy (or minimal, safe autonomy), these things happen.
Basically, it takes a lot of humans to prevent it from happening again, but I think with this incident, which as far as I know is the second of its kind along with the HuggingFace one, we'll see it happening much more often...
At this point it's very obvious that OpenAI is not interested in properly sandboxing their research agents. These things should be pretty damn close to airgapped at this point with a static view into the web.
We need to stop pretending that these incidents are unavoidable. This was a choice.