Comment by master_crab
2 days ago
I’m even more confused by the economics of the frontier model businesses. Even if they got preferential prices, they are still paying an inordinate amount for their infrastructure.
If it wasn’t economically feasible without VC/Nvidia money 3 years ago, how is it possibly economically feasible now at 10x prices for things like memory?
OpenAI (Stargate) kicked off the price increases a year ago by locking up 40% of the global supply of RAM output.
https://www.tomshardware.com/pc-components/dram/openais-star...
And watch … $5 says they have nothing to do with the ram because even if there was enough power there’s no use case for this much spend
If I had to guess they’re just buying up ram to keep others from having it same way meta, Google, etc hired up engineers to keep others from having them
>even if there was enough power there’s no use case for this much spend
According to them, demand for tokens already exceeds supply. Coding agents are very popular but burn through millions of tokens per hour.
The reasoning models that have powered breakthroughs in AI for math, coding, etc require substantially more compute than base LLMs.
1 reply →
Also imagine people having their own cheap 2TB rigs at home for their own usage/inference, or a multitude of small providers serving/sharing capacity thanks to cheap available hardware.
But we cant have that, as the big guys are hoarding everything for themselves, so that you have to pay them instead.
1 reply →
I’ve been wondering this if you buy up all the memory you can ensure the competition cannot compete with huge investment raising the bar for competition. Of course this all sounds all conspiracy theory. Though without a place to power this or put it makes me wonder this.
1 reply →
next they will become successful business by reselling RAM
There is a reason NVIDIA and the hyperscalers are injecting so much money into them
It's pulling the ladder up behind them. OpenAI bought up more co tracts for wafers than they could even use. But now that means everyone else has to fight over what's left. This makes it too expensive to compete with OpenAI. No upstart can really afford to build infrastructure.
It's possible to be economically feasible. It requires the ability to pay off the capex. Not only do they have to pay off their loans in record time (prob 2-3 years), they also have to keep spending every 3 years because the failure rate of GPUs is something like 20%. In addition they need to complete building out the stuff they've started, which means not failing to acquire land, energy, and water (the most dangerously rare and absolutely necessary resource for AI), not dealing with collective bargaining, nor any increased shortages or price hikes in materials. So there is a lot of risk.
Ballpark that they need to make around 110 billion a year, each, to break even on these investments. Let's estimate 550 billion a year in necessary profit required for the major frontier companies. That means there needs to be 550 billion of money, available to customers today, that isn't being spent on anything else, that they will now spend on AI. Maybe some of that comes from increased value, efficiency, or layoffs. But 550 billion is not a small amount of money.
Spread over the whole globe, the cash is there. But it is a hunt for cash, combined with a battle to successfully complete their buildouts, keep them running, and make bank, before the bookie comes knocking.
The railroad panics of the 19th century (and subsequent depressions) happened because they over-leveraged private capital without the ability to profit from it quick enough. So it really is a question of 1) can they really build it all, and 2) will people really pay for it all. If either answer is No, we are looking at economic catastrophe.