Comment by anon7000

I don’t really know how these training runs operate in reality. But I assume it’s using a lot of raw GPU power directly. It’s hard for me to visualize how exactly you’d go about completely cutting off these datacenter and cloud resources from the internet without actually going there, unplugging the WAN connection, and physically typing out what you need to happen on the cluster.

It seems like whatever virtualized sandboxes they have are not enough. But it’s equally hard to imagine their SWEs jumping on a plane to a data center to do this work locally