Comment by sixothree
3 months ago
Year over year gains in computing continue to slow. I think we keep forgetting that when talking about these things as assets. The thing controlling their value is the supply which is tightly controlled like diamonds.
They have a fairly limited lifetime even if progress stands still.
Last I checked AWS 1-year reserve pricing for an 8x H100 box more than pays for the capital cost of the whole box, power, and NVIDIA enterprise license, with thousands left over for profit. On demand pricing is even worse. For cloud providers these things pay for themselves quickly and print cash afterwards. Even the bargain basement $2/GPU/hour pays it off in under two years.
Labor! You need it to turn the bill of sale into a data center and keep it running. The bargain basement would be even cheaper otherwise...
Honestly, I don't fully understand the reason for this shortage.
Isn't it because we insist on only using the latest nodes from a single company for manufacture?
I don't understand why we can't use older process nodes to boost overall GPU making capacity.
Can't we have tiers of GPU availability?
Why is Nvidia not diversifying aggressively to Samsung and Intel no matter the process node.
Can someone explain?
I've heard packaging is also a concern, but can't you get Intel to figure that out with a large enough commitment?
> Isn't it because we insist on only using the latest nodes from a single company for manufacture?
TSMC was way ahead of anyone else introducing 5nm. There's a long lead time porting a chip to a new process from a different manufacturer.
> I don't understand why we can't use older process nodes to boost overall GPU making capacity.
> Can't we have tiers of GPU availability?
NVidia do this. You can get older GPUs, but more performance is better for performance sensitive applications like training or running LLMs.
Higher performance needs better manufacturing processes.
> Year over year gains in computing continue to slow.
This isn't true in the AI chip space (yet). And so much of this isn't just about compute but about the memory.
From a per mm2 performance standpoint things absolutely have slowed considerably. Gains are primarily being eked out via process advantage (which has slowed down) and larger chips (which has an ever-shrinking limit depending on the tech used)
Chiplets have slowed the slowdown in AI, but you can see in the gaming space how much things have slowed to get an idea of what is coming for enterprise.