Comment by justinnk
2 days ago
Exactly. If someone works on bioengineering viruses that could start a global pandemic, they have to ensure a highly secure working environment. Nothing must ever escape the lab unintentionally. It’s basically common sense. Similar standards should be held when doing such experiments with computer programs that are capable of causing global damage. It must physically be impossible to send anything to the internet.
Back in the 00s: "It's really easy to box an AI, just put it in an airgapped machine and refuse to let it out"
2026: "Oops"
This isn’t escaping in the same sense- the model was executing within the OpenAI infra. If it ported its entire architecture/weights into a public cloud to survive being turned off… that’d be pretty cool.
Recall that the Morris Worm was designed as a harmless proof of concept, but ended up taking down 10% of the internet. Exponential growth can quickly get out of control. You would think that people would've learned that lesson from COVID.
I wonder how these companies airgap the weights while allowing prompts to come in and outputs to come out.
You pretty much get this for free. The box doing the inference has one job, to do inference - there's no reason for it to interpret the inputs/outputs in any way, so as long as it doesn't do something stupid like modify the output stream and echo out the model weights, you're fine.
Content filtering, interpreting tool calls, etc can all happen downstream on boxes that don't have access to the weights.
3 replies →
Are we thinking of a situation a few years back with a certain type of research into bat viruses?
Are you conflating that with the radioactive spider incident? The bat was just some weird rich guy trying to be tough I think. Probably Elon.