Comment by zahlman
7 hours ago
I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think.
I still think this is a sign that they are not taking their own rhetoric seriously.
7 hours ago
I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think.
I still think this is a sign that they are not taking their own rhetoric seriously.
> I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think.
These labs are one of the most valuable and heavily funded enterprises in the whole world, that they can't properly air-gap their systems to me reads as if they "agents" and LLMs are not as good as they say they are, because if they were, why would it be hard/expensive to air gap a system? They already scraped most if not all of the internet, where did that data go?
it's really weird to hear frontier labs say "our internal models are basically AGI" while also saying "airgapping is too hard uwu".
if your internal models are so damn good, they should be able to "one shot" airgapping... right?
Agents need packages like the rest of us. Ruby gems, npm packages, Maven, pip, docker images..
Not surprised this is always what they have and hack.
Who would use an Agent that spends $10,000 re-implementing some OAuth lib or reverse-engineering a proprietary lib when it's free on the internet?
You don't need a full air gap. Set up a microVM with network access limited to local network and send all package requests through a filtering gateway that only allows normal download endpoints. Or self host a big collection of popular packages if you need extra security.
Isn't that exactly what they did? The bots could only access the jfrog instance, so they hacked jfrog?
2 replies →
There was and continues to be no reason to share the package manager between models. This was begging for abuse.
> Agents need packages like the rest of us. Ruby gems, npm packages, Maven, pip, docker images..
Yes, yes they do, but read through artifact proxies are dodgy as fuck, which is why and facebook (and I assume a fuckload others) don't have them.
Also semi-airgapped labs are a lot less expensive than you think at that scale. Once you have to do multi-region VLANs with machine certs before you get access to juicy VLANs, the difference between "no internet for you" and "mostly airgapped" falls to almost zero.
Also I would want an artifact mirror because a) that give a good signal about how the model reacts, and what training material its latched onto, b) it hides what the models are doing from the outside.
Not that airgapping is expensive so much as it's really, really inconvenient once you take it seriously. You need to build special rooms for it, you can't just API out to a datacenter. You need to have processes for requesting data be sent into the box. And so on.
I feel like there is a reasonable compromise between "yeah they have full internet access" and "separate airgapped rooms that require multiple levels of authorization to access" that would make this a lot better without that much more work. I feel like they're doing it intentionally to show how dangerous these models are and that the government must step in and protect them
It's expensive if it wasn't part of the planning and design. The same as 'security' is expensive, or compliance with regulations is expensive.
It is also a choice to not do any or all of the above.