Comment by sailingparrot

5 days ago

> they also clearly failed to contract specialists like myself to advise them on how to airgap software properly

Why would they want to airgap it though? They are trying to evaluate the model capabilities, alignment, potency etc. A model which will not run in an airgapped environment in prod.

So if you run your evals in airgapped environment, sure, the model doesn't bother breaking out of it's isolation and doesn't attack HF. But you have no idea what will happen once you release it in prod with internet connection, so are you in any way better off?

I would much rather have this happen while there is a single instance of the model running in a fairly well monitored environment, than when it's processing thousands of requests per second for real users, some with dubious motives, some with credentials right there on their laptop, some using it inside government facilities etc.

6 comments

sailingparrot

Reply
lrvick  5 days ago

You can simulate the internet in an airgapped environment for the tests and services you want it to interact with if you have enough disk space, and, they absolutely do.

  • sailingparrot  5 days ago

    Sure, I'm not saying it's an untractable problem. But it's not as dumb as "how come a team of engineers paid millions haven't even heard of airgapping".

    Creating a fake internet-like environment good enough to trick an advanced model that is very good at finding intricate flaws is not a simple job. Especially given that models can decide to behave differently once they suspect they might be under evaluation [1], which a fake internet would absolutely give away.

    [1]: https://www.anthropic.com/research/alignment-faking

    • lrvick  5 days ago

      I am not saying responsible research is easy, but I am saying that at their budget and scale there are no excuses to cut major corners on safety when the stakes and potential for very bad outcomes for others are this high.

anematode  5 days ago

For security??

Why else would they purposely lower cyber refusals in that case.

  • sailingparrot  5 days ago

    > For security??

    Really? By doing that you increase your security during eval and drastically lower your security at prod time, where internet is accessible, and where you are running much much much more requests in parallel, making it much harder to spot the one thats going rogue, in all kind of critical environment on potentially risky requests.

    > Why else would they purposely lower cyber refusals in that case.

    Because safety guardrails sometime fail, classifier misclassify, or can be inadvertently turned off by a bad PR etc. You can also imagine a more intelligent model working to go around guardrails by decomposing its actions into smaller ones that appear non-threatening to the safety classifier which does not have the entire context.

    If you are going to deploy the system with internet access, you better be certain that you know what the worst case scenario WITH internet access looks like.