Comment by lumost
17 days ago
This is true if you take the ai market as equal to the market for labor discounted to 5-10% penetration.
It’s not a totally unreasonable assertion, it’s the implication of the assertion that we are uncomfortable with. There is no reason for the models to stop their improvements in the near future.
> There is no reason for the models to stop their improvements in the near future.
Sure there is.
1. The cost of each new generation of training runs appears to be rapidly rising
2. The Trump admin just told the leading model to stop making it available to non-Americans, which in practice meant stop providing it at all
3. The factories to make the hardware are hitting bottlenecks, and while they've currently been navigated around, there's never a guarantee the next one will be
Currently I'm wondering at what point the direct impact on the US energy supply gives the US a taste of Baumol's cost disease as AI companies continue to outbid everyone else for electricity.
There are some counters to this, especially in electricity. We'll see massive expansions of wind and solar in the US because of this. Both the speed of install and low costs will guarantee it.
> We'll see massive expansions of wind and solar in the US because of this. Both the speed of install and low costs will guarantee it.
Implausible while Trump remains in office. He hates renewables, shuts them down even when doing so actively costs money.
Between AI hallucinated content and the politicisation of the numbers, I'm not sure how much AI compute capacity is being planned right now; would you accept a claim of 300 GW? It's a number I heard recently.
Given the capacity factor of PV, even China would have to think carefully before supplying that much PV over the next few years (300 GW avg ~= 3TW nameplate).
(Not sure about wind, wind's CF seems to vary between years).
2 replies →
As long as Chinese companies keep pushing on, so must US companies too.
It would not surprise me at all if we suddenly start seeing top US AI companies lobby against Chinese models, or even the gov. making it illegal to use Chinese AI models.
But in this day and age, I just don't think it is possible. A distant third option would be that the big AI companies try to make hardware so expensive that people simply can't run their own models, while blocking access to foreign models.
> There is no reason for the models to stop their improvements in the near future.
You speak as if "improvements to models" is just function of time, and resources are infinite.
Models keep improving as long as there are resources to allow for larger and larger datacenters, if we hit a scientific breakthrough once LLM technology become the bottleneck, if the economy is infinite to allow infinite growth, and (geo)politics is not a thing to worry about. Or we discover ASI, machine improve themselves and we reach the technological singularity.
I know everybody is drinking the kool aid by the gallon, but can we maintain a little bit of objectivity?
yeah, it's funny how so many think the beginning of the S-curve is an exponential.
Granted, we don't know when the S-curve will inflect, but predicting too great an outcome is just as silly as discounting it altogether.
The s curve won’t inflect until it becomes difficult to allocate additional resources due to economic limitations. There is no sign that training a model on 10x the compute won’t lead to at least an equivalent improvement as the last order of magnitude increase.
If we define the Pareto frontier’s input in terms of a magic “compute equivalent unit”. We get a free order of magnitude from nvidia hardware improvements every 2-3 years. We get another order of magnitude from capital expenditure every 6-12 months. Kernel improvements to the models themselves likely yield an order of magnitude gain at some periodicity.