Comment by pianopatrick
1 hour ago
We should maybe apply the same kinds of physical controls to this software security research as we do to gain of function biological research.
I.e. do not do these frontier model training runs on virtual machines in the cloud which depend on software systems to prevent internet access. Do them in a special data center that is physically disconnected from the outside world. Like a data center in a faraday cage under a mountain powered by a generator with no cables coming in or out.
Sure, but in order to know security standards you have to test and validate. Everyone accepts some standard as 'secure' after rigorous testing and validation.
I think we're learning the standards for sandboxxing change when you consider AI's ease in discovering exploits.
A sandbox good enough to contain human generated code and intent, is likely not good enough for AI generated code and intent (seems obvious to say that but).
And so poking on these issues, regardless of who asked to escape the box, is still worthy research.
I don't mind the research. I just don't trust any software system to contain these AIs. I would prefer we use hardware and physics to contain them and not depend on software at all.