Comment by master_crab

> People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model...

It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one.

That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).

I think what will keep the industry afloat, all else failing, is the surveillance industry! Nothing like a fat reoccurring cheque from the government to check if little Jimmy is committing thought crime!

LLMs needing less compute would actually be a good thing for Nvidia due to Jevons paradox. Right now token costs are an impediment to using AI more broadly, and more efficient models would help adoption in cases where AI has proven to be useful, like coding.

https://en.wikipedia.org/wiki/Jevons_paradox

That's unless the code produced in the future is much more complex than today's.

You're not considering video which OpenAI opted out of when they retired Sora.

Generative video requires significantly more computing power and energy than generative text.

OpenAI is fucked, compute is still needed, it's just them that isn't.

I've been thinking about that and that's why Nvidia's prices are surprising to me. Investors should know that better than me so there must be something I don't know

This is the right kind of analysis, but we can look broader. Both the demand and supply situations are a lot more extreme and dynamic than appears at first glance. E.g. to your points:

1. Yes, smaller models will become more popular, especially as the tokenmaxxing trend dies down and people start stretching their budgets farther. That is a downward pressure on demand.

But along the same dimension, consider that currently only about 40 - 60% of the world uses AI for only about 5 - 15% of their work hours. That means there is still 2x growth from users and 7x - 20x growth from the rest of the work hours left to capture! That is 14x - 40x more demand. Then consider that agentic tasks require multiples more tokens, and that is the kind of usage that is most likely to be deployed, and also the kind of usage that is the least used right now. That's another huge multiple to be tacked on.

And the entire AI industry has been lamenting the extreme compute crunch they're facing (and also why Claude has 9's comparable to GitHub; whereas OpenAI has been chugging along because Altman was OK being called a "podcasting bro" while desperately scrounging for compute years in advance.)

Nvidia's meteoric rise is entirely due to this kind of exploding demand with extremely limited supply.

2. Competing hardware is definitely a threat, but it has its own hurdles. Because the real bottleneck is not Nvidia, it's TSMC.

Pretty much all demand for all chips in all devices in all the world flow to, like, 3 companies in the world that actually fabricate them, and TSMC is the biggest. And the supply is extremely tight, as the exploding costs of electronics clearly shows.

So now TSMC will of course try to keep all its customers happy, but it will inevitably be forced to choose which ones it will keep happiest. And those will be the customers who can pay it the most. And that would be the one with all the money from its de facto status as a monopoly (and possibly even a monopsony)...

Which would be Nvidia ;-)

So yes, compute per task is falling rapidly... but it's barely a dent in the humongous total addressable demand, and the amount of hardware to support that compute is still very constrained, and most of that supply will likely flow through Nvidia.

They need the right harness and either your help it auto produces in time enough content to further improve.

If you reshuffle your argument, and apply the same facts you get to a similar conclusion but with a drastically different spin.

> it's more specialization

China, constrained by hardware, and talent (not to slight the Chinese, but they are limited to domestic resources - and much of the US effort is very international). They did, what the Chinese do, and optimized the process of production, and drastically lowered the cost of development of their models. Cheeper to build, cheaper to run is just good economics.

Meanwhile in the us, we have open AI doing "experiments" - it looks like the costs around the hugging face hack are going to be about the same as China would spend on building out one of their smaller efforts (several million dollars). (Depending on whos numbers you trust, the fact that I can even make this claim should make you raise an eyebrow).

Go back to the 80s' and "expert systems" - most people will tell you that for their time, they were amazing, and useful. People would have loved to have more of them but they were so cost prohibitive that we all but abandoned them for serious use. The US frontier labs seem to have forgotten this lesson and their calls to "slow down" look like an excuse to "cut the waste so we can move to making money".

> investor money

But that’s the point. Investors believe investment in AI will pay off.

That is why I added the last line.

But I also doubt Nvidia is on the hook if OpenAI just no longer wants the compute. I bet they are only on the hook if OpenAI cannot pay for it (is insolvent in some way).

I also have to bring up that OpenAI has already spat out an inference chip that beats Nvidia on flops per watt. So they could potentially not need the compute while other ai companies do.

To make things worse, hardware prices have spiked, due to AI companies.

Fairly sure data center construction costs are also going up (they require so many resources that everything is constrained at the moment, especially electricity production).

So I don't understand in what world these frontier AI companies can somehow become profitable. The basic tech they're using is basically the same. Yes, around the edges there are a lot of things that can be done, and were done, like caching, batching, mixture of experts, etc, but basically everyone has done all of that by now, and they're still losing money.

So:

Total costs going up a lot - revenues per unit not increasing proportionally, if anything, Chinese models are forcing those down.

How does that math work out to profits? I don't see it.

Or about as bad, after trillions of dollars in investments over multiple years, let's say the entire frontier AI sector has a total profit of $20bn by 2030. In what world does that make sense? Assuming they can scale that total profit to $100bn in 2035 without investing another cent from 2027 to 2035 (utterly ridiculous), the return on investment would happen in roughly 20 years.

Do you use coding agents? Just curious bc from my experience using coding agents, open ai’s codex is neck and neck with anthropic’s claude code if not ahead. I wouldn’t agree that OpenAI hasn’t done anything since ChatGPT since codex is my daily driver for software engineering

To each their own. When OpenAI droped Sora and focused more on Codex, the product improved dramatically and I'm probably not the only one who dumped Claude Code subscription in favor of Codex; OpenAI's is miles ahead of Anthropic and has been since at least the release GPT 5.6 Sol - even the PR and generous resets is far better than how Anthropic is nickel and diming by requiring paid subscribers to pay yet more credits to even use their best available model (which is not even as good as OpenAI's top 2 models)

I've paid more money to OpenAI than I've spent on all other products I pay for combined the past 5 years.