During the Covid global chip shortage Intel announced new factories to address the production bottleneck, before the factories even started to be constructed, the shortage was long over and the projects were eventually canceled.
What if the AI companies start selling the hardware with their models?
I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.
That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.
I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.
Samsung is already building another fab (which was originally suspended due to low memory prices years ago). Hopefully when it goes live in a few years it's not all dedicated to HBM for AI.
During the Covid global chip shortage Intel announced new factories to address the production bottleneck, before the factories even started to be constructed, the shortage was long over and the projects were eventually canceled.
It will be just in time for when AI will run very well on consumer hardware and the need for data centers will collapse.
What if the AI companies start selling the hardware with their models?
I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.
Sound familiar?
More likely they would sell a chip with the model etched in silicon for the dramatically higher token/s.
1 reply →
Most likely those models would be extracted from the hardware in no time.
1 reply →
There will be demand for both
That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.
I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
4 replies →
This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price
I'm not sure if this prediction will hold true.
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
17 replies →
It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.
8 replies →
Or even more amusing in time for the bubble to burst, I hope the big RAM makers end up holding the whole bag for that. Greedy bastards.
Nearly the entire reason we're in this mess is because RAM/SSD/HDD makers are terrified of AI bubble bursting.
Long-term contracts, not building new fabs - they're in full on hedging mode right now.
Since you’re not greedy when there is a memory glut I’m sure you’ll be willing to pay extra to make it fair.
7 replies →
Samsung is already building another fab (which was originally suspended due to low memory prices years ago). Hopefully when it goes live in a few years it's not all dedicated to HBM for AI.
Not even close.
Not until the industry gets severe indigestion, which doesn't seem that far off.