Comment by iLoveOncall
17 hours ago
> short of major algorithmic breakthroughs I am not convinced the global demand for GPUs will drop any time soon
Or, you know, when LLMs don't pay off.
17 hours ago
> short of major algorithmic breakthroughs I am not convinced the global demand for GPUs will drop any time soon
Or, you know, when LLMs don't pay off.
Even if LLMs didn't advance at all from this point onward, there's still loads of productive work that could be optimized / fully automated by them, at no worse output quality than the low-skilled humans we're currently throwing at that work.
inference requires a fraction of the power that training does. According to the Villalobos paper, the median date is 2028. At some point we won't be training bigger and bigger models every month. We will run out of additional material to train on, things will continue commodifying, and then the amount of training happening will significantly decrease unless new avenues open for new types of models. But our current LLMs are much more compute-intensive than any other type of generative or task-specific model
Run out of training data? They’re going to put these things in humanoids (they are weirdly cheap now) and record high resolution video and other sensor data of real world tasks and train huge multimodal Vision Language Action models etc.
The world is more than just text. We can never run out of pixels if we point cameras at the real world and move them around.
I work in robotics and I don’t think people talking about this stuff appreciate that text and internet pictures is just the beginning. Robotics is poised to generate and consume TONS of data from the real world, not just the internet.
1 reply →
Inference leans heavily on GPU RAM and RAM bandwidth for the decode phase where an increasingly greater amount of time is being spent as people find better ways to leverage inference. So NVIDIA users are currently arguably going to demand a different product mix when the market shifts away from the current training-friendly products. I suspect there will be more than enough demand for inference that whatever power we release from a relative slackening of training demand will be more than made up and then some by power demand to drive a large inference market.
It isn’t the panacea some make it out to be, but there is obvious utility here to sell. The real argument is shifting towards the pricing.
> We will run out of additional material to train on
This sounds a bit silly. More training will generally result in better modeling, even for a fixed amount of genuine original data. At current model sizes, it's essentially impossible to overfit to the training data so there's no reason why we should just "stop".
4 replies →
How much of the current usage is productive work that's worth paying for vs personal usage / spam that would just drop off after usage charges come in? I imagine flooding youtube and instagram with slop videos would reduce if users had to pay fair prices to use the models.
The companies might also downgrade the quality of the models to make it more viable to provide as an ad supported service which would again reduce utilisation.
For any "click here and type into a box" job for which you'd hire a low-skilled worker and give them an SOP to follow, you can have an LLM-ish tool do it.
And probably for the slightly more skilled email jobs that have infiltrated nearly all companies too.
Is that productive work? Well if people are getting paid, often a multiple of minimum wage, then it's productive-seeming enough.
1 reply →
Who is generating videos for free?
Exactly, the current spend on LLMs is based on extremely high expectations and the vendors operating at a loss. It’s very reasonable to assume that those expectations will not be met, and spending will slow down as well.
Nvidia’s valuation is based on the current trend continuing and even increasing, which I consider unlikely in the long term.
> Nvidia’s valuation is based on the current trend continuing
People said this back when Folding@Home was dominated by Team Green years ago. Then again when GPUs sold out for the cryptocurrency boom, and now again that Nvidia is addressing the LLM demand.
Nvidia's valuation is backstopped by the fact that Russia, Ukraine, China and the United States are all tripping over themselves for the chance to deploy it operationally. If the world goes to war (which is an unfortunate likelihood) then Nvidia will be the only trillion-dollar defense empire since the DoD's Last Supper.
China is restricting purchases of H200s. The strong likelihood is that they're doing this to promote their own domestic competitors. It may take a few years for those chips to catch up and enter full production, but it's hard to envision any "trillion dollar" Nvidia defense empire once that happens.
4 replies →
> short of major algorithmic breakthroughs I am not convinced the global demand for GPUs will drop any time soon
>> Or, you know, when LLMs don't pay off.
Heh, exactly the observation that a fanatic religious believer cannot possibly foresee. "We need more churches! More priests! Until a breakthrough in praying technique will be achieved I don't foresee less demand for religious devotion!" Nobody foresaw Nietzsche and the decline in blind faith.
But then again, like an atheist back in the day, the furious zealots would burn me at the stake if they could, for saying this. Sadly no longer possible so let them downvotes pour instead!
They already are paying off. The nature of LLMs means that they will require expensive, fast hardware that's a large capex.
They aren’t yet because the big providers that paid for all of this GPU capacity aren’t profitable yet.
They continually leap frog each other and shift around customers which indicates that the current capacity is already higher than what is required for what people actually pay for.
Google, Amazon, and Microsoft aren’t profitable?
7 replies →
Where? Who’s in the black?