Comment by aviinuo
10 hours ago
Nvidia GPUs let you load encrypted weights onto them and to decrypt on chip so Google would need to have some sort of Nvidia management firmware attack to read from HBM while the GPU is live
10 hours ago
Nvidia GPUs let you load encrypted weights onto them and to decrypt on chip so Google would need to have some sort of Nvidia management firmware attack to read from HBM while the GPU is live
Distillation does not require access to weights.
Isn't distillation more of a black box type affair?
Doesn’t really matter what happens at the GPU level when Google can see all API traffic.
Except google cloud doesn't use Nvidia GPUs but TPUs.
Google has both. Afaik we don't know whether OpenAI models will run on TPUs or GPUs here.
I bet it would be the NVIDIA, at least initially, not for the encryption capability per se, but for simple fact that adapting to TPU at high performance is non-trivial.