Costs will plummet as better hardware becomes available and priced reasonable so that people can more easily run their own open models locally. But that won't help Antropic/OpenAI make more money, quite the opposite.
Algorithms are also improving. I believe it's very unlikely for these two improvements together to not result in one to two orders of magnitude cheaper cost per "intelligence". Of course, that might just make use cases that are too expensive today viable and thereby increase usage further.
Yeah, it’s called supply and demand. Demand for memory went way up suddenly. Now supply is going up rapidly as companies try to cash in on that demand.
Supply will eventually catch up with demand. Then the prices will come back down.
A lot of the new hardware requires retrofitting existing datacenters for appropriate cooling, or is waiting to be installed because the new datacenters haven't been built yet. By the time they're installed it's likely a lot of Blackwell GPUs are going to be very out of date. Newer hardware is turning into huge capex bills along with the corresponding depreciation costs. Basically, it's not the same as plugging a new GPU into your desktop, the upfront investment is extremely expensive and all the numbers I'm seeing suggest that the newer GPUs are costing more to run, not less.
Costs will plummet as better hardware becomes available and priced reasonable so that people can more easily run their own open models locally. But that won't help Antropic/OpenAI make more money, quite the opposite.
That trend held true for the past three years, but it doesn't feel as safe to me now.
But memory costs are going way up. And both OpenAI and Anthropic bumped up the price of their frontier models in April.
Algorithms are also improving. I believe it's very unlikely for these two improvements together to not result in one to two orders of magnitude cheaper cost per "intelligence". Of course, that might just make use cases that are too expensive today viable and thereby increase usage further.
Yeah, it’s called supply and demand. Demand for memory went way up suddenly. Now supply is going up rapidly as companies try to cash in on that demand.
Supply will eventually catch up with demand. Then the prices will come back down.
A lot of the new hardware requires retrofitting existing datacenters for appropriate cooling, or is waiting to be installed because the new datacenters haven't been built yet. By the time they're installed it's likely a lot of Blackwell GPUs are going to be very out of date. Newer hardware is turning into huge capex bills along with the corresponding depreciation costs. Basically, it's not the same as plugging a new GPU into your desktop, the upfront investment is extremely expensive and all the numbers I'm seeing suggest that the newer GPUs are costing more to run, not less.
Sure, with all the component shortages it’s not surprising the current GPUs are coming at a massive premium.
Eventually either the supply will go up or companies will start buying fewer overpriced GPUs.
Either way, the price per token will come down as hardware improves and supply and demand reach equilibrium.
Technological obsolesence is a bitch eh.