Comment by keeda

This is the right kind of analysis, but we can look broader. Both the demand and supply situations are a lot more extreme and dynamic than appears at first glance. E.g. to your points:

1. Yes, smaller models will become more popular, especially as the tokenmaxxing trend dies down and people start stretching their budgets farther. That is a downward pressure on demand.

But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. That means there is still 2x growth from users and 7x - 20x growth from the rest of the work hours left to capture! That is 14x - 40x more demand. Then consider that agentic tasks require multiples more tokens, and that is the kind of usage that is most likely to be deployed, and also the kind of usage that is the least used right now. That's another huge multiple to be tacked on.

And the entire AI industry has been lamenting the extreme compute crunch they're facing (and also why Claude has 9's comparable to GitHub; whereas OpenAI has been chugging along because Altman was OK being called a "podcasting bro" while desperately scrounging for compute years in advance.)

Nvidia's meteoric rise is entirely due to this kind of exploding demand with extremely limited supply.

2. Competing hardware is definitely a threat, but it has its own hurdles. Because the real bottleneck is not Nvidia, it's TSMC.

Pretty much all demand for all chips in all devices in all the world flow to, like, 3 companies in the world that actually fabricate them, and TSMC is the biggest. And the supply is extremely tight, as the exploding costs of electronics clearly shows.

So now TSMC will of course try to keep all its customers happy, but it will inevitably be forced to choose which ones it will keep happiest. And those will be the customers who can pay it the most. And that would be the one with all the money from its de facto status as a monopoly (and possibly even a monopsony)...

Which would be Nvidia ;-)

So yes, compute per task is falling rapidly... but it's barely a dent in the humongous total addressable demand, and the amount of hardware to support that compute is still very constrained, and most of that supply will likely flow through Nvidia.