Comment by filleokus

> Will all these GPUs be used for inference once a SOTA model's training checkpoint/batch is done?

I have no real data to back this up, but that has always been my assumption.

Claude says that K3 can be assumed to have required 10-100M GPU hours. If you have 100k GPUs that would mean like 6 weeks of training. 100k GPU's can serve 3-30 trillion tokens of K3 per day. Google apparently serves ≈100 trillion per day [0].

The big labs probably want to have capacity to fairly quickly train / post train different SOTA models continuously + being able to serve peak inference demand in valuable markets (US daytime?).

[0]: https://blog.google/innovation-and-ai/sundar-pichai-io-2026