Comment by veunes
2 months ago
The funniest thing about this story is that NVIDIA has essentially become a TPU company. Look at the Hopper and Blackwell architectures: Tensor Cores are taking up more space, the Transformer Engine has appeared, and NVLink has started to look like a supercomputer interconnect. Jensen Huang isn't stupid. He saw the threat of specialized ASICs and just built the ASIC inside the GPU. Now we have a GPU that is 80% matrix multiplier but still keeps CUDA compatibility. Google tried to kill the GPU, but instead forced the GPU to mutate into a TPU
There's an issue with building a swiss knife chip that supports everything back to the 80s, it works great until it doesn't (Intel)
Timing is everything here. The swiss army knife approach only loses when tasks stop changing. Intel suffered when workloads like web and mobile stabilized
In AI, we're still in the explosion phase. If you build the perfect ASIC for Transformers today, and tomorrow a paper drops with a new architecture, your chip becomes a brick. NVIDIA pays the "legacy tax" and keeps CUDA specifically as insurance against algorithm churn. As long as the industry moves this fast, flexibility beats raw efficiency