Comment by keeda
4 months ago
I feel like this bubble actually has two bubbles happening:
1. The infra build out bubble: this is mostly the hypescalers and Nvidia.
2. The AI company valuation bubble: this includes the hyperscalers, pure-play AI companies like OpenAI and Anthropic, and the swarm of startups that are either vaporware or just wrappers on top of the same set of APIs.
There will probably be a pop in (2), especially the random startups that got millions in VC funding just because they have a ".ai" in their domain name. This is also why the OpenAI and Anthropic are getting into the infra game by trying to own their own datacenters, that may be the only moat they have.
However, when people talk about trillions, it's mostly (1) that they are thinking of. Given the acceleration of demand that is being reported, I think (1) will not really pop, maybe just deflate a bit when (2) pops.
OK, that is interesting. Separating infra from AI valuation. I can see what you mean though because stock prices are volatile and unpredictable but a datacenter will remain in place even if its owner goes bankrupt.
However, I think the AI datacenter craze is definitely going to experience a shift. GPU chips get obsolete really fast, especially now that we are moving into specialised neural chips. All those datacenters with thousands of GPUs will be outcompeted by datacenters with 1/4th the power demand and 1/10th the physical footprint due to improved efficiency within a few years. And if indeed the valuation collapses and investors pull out of these companies, where are these datacenters supposed to go? Would you but a datacenter chock full of obsolete chips?
Right, the obsolence rate of GPUs is one of the primary drivers of the depreciation shenanigans aspect of the bubble.
However, I've come across a number of articles that paint a very different picture. E.g. this one is from someone in the GPU farm industry and is clearly going to be biased, but by the same token seems to be more knowledgeable. They claim that the demand is so high that even 9-year old generations still get booked like hot cakes: https://www.whitefiber.com/blog/understanding-gpu-lifecycle
> They claim that the demand is so high that even 9-year old generations still get booked like hot cakes
What does this prove? Demand is inflated in a bubble. If the AI company valuation bubble pops, demand for obsolete GPUs will evaporate.
The article you're linking here doesn't say what percentage of those 9-year-old GPUs already failed, nor does it say when they were first deployed, so it's hard to draw conclusions. In fact their math doesn't seem to consider failure at all, which is highly suspicious.
In another subthread, you pointed to the top comment here about a 5-year MTBF as supposedly contradicting the original article's thesis about depreciation. 5 years is obviously less than the 9 years here, so clearly something doesn't add up. (Besides, a 5-year MTBF is rather poor to begin with, and there isn't normally a correlation between depreciation and MTBF. So this is not a smoking gun which contradicts anything in Tim Bray's original article.)
2 replies →