Comment by nrmitchi
2 days ago
If you are attempting to run exercises like this, it is wildly negligent to not be running it in a physically-airgapped environment (potentially with a physical power shutdown).
You can not tell me that OpenAI doesn’t have the resources or ability to run tests like this in a physically-non-networked environment w/ sufficient compute for its needs.
If it was just one test, sure. But if they're spinning these up continuously with new models on tens of thousands of GPUs, air gapping becomes impractical. I would mostly fault them on having no guardrails at all. They should have a monitor/external harness that looks for successful access to external networks then stop it there. They may as well let the models test their own networks for vulnerabilities. That's going to be really important to have going forward.
You can have large scale airgapped environments.
They don’t even need to be fully airgapped from each other (and is not what I’m suggesting).
But there should be no physical (physical layer; wireless counts) to the internet.
Can models detect they are airgapped and change their behaviors?
How much of the internet do you have to simulate to know if the model knows it's in training?
3 replies →
If its worth doing for one test, its worth doing when you spin up many. Quantity of tests does not change the value of an airgap.
Well, then when it detects an air-gapped environment it will just behave differently. I feel like we underestimate in general the way agents behavior changes when the environment changes. Related: "power corrupts"
> wildly negligent to not be running it in a physically-airgapped environment
Why should it be physically airgapped? Clients won't be doing that.
Clients are not using it with security guardrails disabled. If you want to run it with all the safeties turned off, you don’t run it somewhere it can escape.
Did we learn nothing from all of those Star Trek holodeck jailbreaks?
Hard disagree. You have to assume security guardrails can be by bypassed or will fail to detect an attacker. So if you are going to deploy this model in production non-airgapped, you better know how it will behave in this non-airgapped environment without guardrails.
And if you are too afraid to test it without guardrails, that probably means it shouldn’t be released.
Because they are testing it and are expected to erect guardrails before releasing.
What are the specific guardrails implemented after the verification/testing phase of development?
Is it safe to release such software if it has only been tested in environments where certain major risk areas do not exist?
1 reply →
Because by the time it goes out to clients it (should be) thoroughly tested and aligned for safety, but at the present moment it isn't.
Being that proving alignment is impossible in these models they can never be released.
How else would they put out news about their "rogue super smart AI"
This is infuriating. You are talking about people who have stolen and monetized the entirety of mankind's knowledge in plain view of everyone, and they still haven't faced a shred of consequences. Of course they don't go about doing things ethically or responsibly