Comment by mrinterweb
17 hours ago
Investing in AI infrastructure seems very risky to me. With how short of a lifecycle computing hardware generally has, does it make sense to blow billions on hardware that will need to be replaced in a few years? AI companies have been burning billions on hardware. Will the costs of that hardware be covered by profits before the hardware lifecycle is up? That's a big gamble.
We used to talk about AI as having no moat (easy for other players to accomplish similar AI achievements). China has made that clear. Some open weight LLMs are fine and can run on laptops. It seems the new moat is model parameter size and VRAM requirements. I would bet on innovation in hardware or disruptive algorithms changing that game so better LLMs can run on more personal computers. Remember how bitcoin miners all used GPUs then ASICs came along and made that no longer profitable?
There are many ways the AI industry can be disrupted which makes it that much more volatile.
Considering we're two years in and everyone's still... talking about chatbots pretty much. Yea, I'd say that's a hell of a gamble.
Of course the hardware can probably be repurposed towards crypto mining afterwards, so hey!
The chatbot focus may be a news/accessibility bias. Take a look at deepmind's research blog to see applications in more areas.
There's definitely some cool stuff coming out of it!
Is it going to be as world upending as some folk were making it out to be? I'm not sold on that yet.
While I agree with the general sentiment on throwaway compute infra, the generated know-how with large scale experiments is not thrown away. I think a lot hinges on the scaling laws and whether you will hit the jackpot at a certain scale before everyone else. This is hard to guesstimate so someone has to do it in the spirit of empiricism. This might sound a lot like gambling or exploring depending on your sentiment. So, I think it is more justified to criticize the scale and the risks than the spirit of these investments.
While scaling laws are only empirical curve fitting and extrapolation, none of them predict a discontinuous "jackpot effect."
AFAIK, certain abilities such as understanding arithmetic manifest at discrete scale points even though there is a continuous build up of potential. There is also the more remote possibility of a discrete scale that AI takes over its own training or at least starts to contribute substantially. A lot of real world leverage and arbitrage depends on such discrete surprises that may not be visible during the continuous incremental evolution. I think this principle holds computationally as much as it does biologically.
You're assuming that gold is going towards infrastructure. I don't think a lot of it is. I think it's a money grab while the getting's good.
Once you can run a GPT5 level LLM locally on a device, it’s over. All this mighty infrastructure is no longer any more impressive than a top of the line 2013 Mac Pro in 2025. I think we’re 10 years away from that.
10 years from now, the capabilities of gpt5 will be as relevant to AI as Atari is to modern gaming
Unless GPT 5 is more like the PlayStation 5 and not the Atari 2600.
So people will be paying money for the nostalgia of ChatGPT after it dies? That tracks.
1 reply →
I doubt it. Newer state of the art models might be a little better, but not enough to justify paying $1000/month for the average person or employee.
If you can get a GPT5 level AI, locally and privately, for just the cost of electricity, why would you bother with anything else? If it can’t do something, you’d just outsource that one prompt to a cloud based AI.
The vast majority of your prompts will be passing through a local LLM first in 2035, and you might rarely need to touch an agent API. So what does that mean for the AI industry?
Consumer devices are already available that offer 128gb specifically labeled for AI use. I think server side AI will still exist for IoT devices, but I agree, 10 years seems pretty reasonable timelie to buy a GTX 5080-sized card that will have 1TB of memory, with the ability to pair it with another one for 2TB. For local, non-distributed use, GPUs are already more than capable of doing 20+ tokens/s, we're mostly waiting on 512gb devices to drop in price, and "free" LLMs to get better.
Are we constrained by RAM production?
RAM Price per GB Projected to decline at 15% per annum.
That's quite a few years before you'll get double the RAM.
For mobile I'm guessing power constraints matter too.
Have you tried the reasoning mode of Gemini Pro 2.5? https://aistudio.google.com/
It gives me the chills, thinking about when it has 1000x cheaper ~GPU compute.
This a hosted model with closed weights, though.
1 reply →
Very high risk/Insane payoff (maybe).
Pretty much it, albeit I think very few appreciate how high's the risk of leaving with nothing in their hands.
Valuation models probably begin to break down when the potential payoff is control of our solar system as we know it.
OTOH it seems like mere single billions can buy control over the US if you're brazen enough, so perhaps these stock market suckers are overpaying with their trillions.
I wonder how expensive it would be to buy control of PRC in a similarly creative (but very different) way.
Each datacenter iteration that increases the size and power consumption is a stepping stone towards (possibly) recursive self-improvement in AI tech. I think that's the way to frame the absolutely massive bets being made by (only a few) companies. There aren't many companies or nation states in the world that can marshal the resources and talent needed to keep competing on this path.
It's a race to the the most powerful and transformative technology in history.
Or it might all collapse like a house of cards. But worth a shot.
> But worth a shot.
Worth betting the entire US economy on it?
Only a significant portion of it.
No. Nothing is but is that what's happening?
Huge bets were made on the Internet technology in the US and it paid off even with the dot com bubble bursting.
If we don't stay on top of technology then we will simply be passed by China.
People also said the Internet was going revolutionize everything in the 90s and it did. AI is already pretty impressive and we are just in the infancy of the technology.
5 replies →
>I would bet on innovation in hardware or disruptive algorithms changing that game so better LLMs can run on more personal computers.
This is classic jevon's paradox. As efficiency increases so does demand.