Comment by cmrx64
9 hours ago
I sincerely doubt anyone is paying the cost for that in training, the overhead is small but it isn’t negligible and training is when it matters most. https://tee.fail can solve it if they are.
9 hours ago
I sincerely doubt anyone is paying the cost for that in training, the overhead is small but it isn’t negligible and training is when it matters most. https://tee.fail can solve it if they are.
memory encryption is cheap. securing the pathway isn't particularly difficult (it's probably decoupled from the TEE monolith)
for example every TPU/GPU has its own private key and the devs load the weights into it by sending it encrypted weights.
it takes half a percentage point off the top last time i evaluated it (nvidia). you might call that cheap but that’s millions of dollars in a run, and for what, protecting from who? especially when the platforms have been compromised to the point of key leak (which they have).
edit: i just looked up training numbers and the impact is even worse, 20-30% throughput vaporized. yeah, nobody is doing that.