Comment by overgard
18 hours ago
A lot of the new hardware requires retrofitting existing datacenters for appropriate cooling, or is waiting to be installed because the new datacenters haven't been built yet. By the time they're installed it's likely a lot of Blackwell GPUs are going to be very out of date. Newer hardware is turning into huge capex bills along with the corresponding depreciation costs. Basically, it's not the same as plugging a new GPU into your desktop, the upfront investment is extremely expensive and all the numbers I'm seeing suggest that the newer GPUs are costing more to run, not less.
Sure, with all the component shortages it’s not surprising the current GPUs are coming at a massive premium.
Eventually either the supply will go up or companies will start buying fewer overpriced GPUs.
Either way, the price per token will come down as hardware improves and supply and demand reach equilibrium.
Technological obsolesence is a bitch eh.