Comment by windexh8er

There's two things here: 1) you clearly don't understand the argument and 2) LLMs are one of the few technologies that doesn't get any cheaper as it scales (totality, not just the cherry picked inference efficiency argument you've tried to make). In fact it gets more expensive because it scales linearly with demand and resources aren't infinite, as I'd hope you could understand.

Also, training costs are never ending so a model that costs 10s of millions of dollars may never yield a profit based on the hardware spend, training time and lack of inference profits before a better model hits the market.

If you're not living under a rock one knows that data center availability for inference currently has low supply and hardware (GPUs specifically) that have been purchased have nowhere to be run and even if they did there's often a lack of power to supply. Why do you think the entire force majeure has taken place with Oracle as of recent?

The unit price of a fixed slice of yesterday's intelligence may be collapsing (~10x/year) as you've argued, all while the total cost of AI is increasing: training the frontier (2.4x/year), building the infrastructure (+77%/year), enterprise bills (3.2x/year), the electricity (+54%/year in the largest US grid), the components (+400% DRAM), and the macro footprint (92% of GDP growth) is rising at an astronomical rate on every measurable point. Epoch / Stanford clearly stated this years ago and it's only getting worse. But if one can't see we're in one of the largest CapEx bubbles [1] of all time... o_O

Copying and pasting a few lines that represents a miniscule fraction of the LLM conundrum. That'll show 'em!

[0] https://arxiv.org/abs/2405.21015 [1] https://siliconanalysts.com/analysis/hyperscaler-ai-capex-de...