Comment by GoToRO
5 hours ago
It will be just in time for when AI will run very well on consumer hardware and the need for data centers will collapse.
5 hours ago
It will be just in time for when AI will run very well on consumer hardware and the need for data centers will collapse.
What if the AI companies start selling the hardware with their models?
I.e., you get a locked down device with access to their AI and maybe a way to run third party apps. Of course the developers of those apps will have to pay a percentage of their revenue.
Sound familiar?
More likely they would sell a chip with the model etched in silicon for the dramatically higher token/s.
A chip that can only be accessed through their software. With a monthly subscription fee, and other enshittification surprises.
Most likely those models would be extracted from the hardware in no time.
I suspect no, because most of the tokens would be used for its internal reasoning loop, and a cheaper model could be used to obfuscate the output.
There will be demand for both
That's never going to happen. By the time you can run current frontier models on your $10k desktop the frontier will have massively advanced and people will want those models instead.
It's going to happen very soon, which is why these frontier labs are scrambling to shut down open source language models. There's an existential risk threatening their obscene returns.
I feel like cost competitiveness of local has been going down, if anything, not up. API providers can use hardware more and have scale efficiencies. Do you see any reason this will reverse?
It is for that reason they are being archived and torrented as a very large middle finger.
> It's going to happen very soon
Why? You can't just assert it. There are very good reasons to think it won't happen soon, and you've given no reasons to think it will happen soon.
3 replies →
I’m not so sure about that. Already AI vendors are back to cutting prices to try and keep customers from cutting back on their usage. My own employer is working hard at pivoting to much smaller fine-tuned models for established use cases, and seeing model performance improvement in addition to large inference cost reductions. Being able to run them locally hasn’t exactly been a disaster for devex, either.
It may turn out that demand for SOTA frontier models isn’t so limitless after all.
Every product follows demand curves. At a price of 0 you could find infinite usage. This has nearly zero relation to how much it costs to provide the product.
4 replies →
This + your ROI in $10k device will be always lower than busy datacenter, its literally math. They sell free compute to others when you dont use it, you will never sell at that level or even you magically sell home compute, you will not compete at price
I'm not sure if this prediction will hold true.
We're not seeing the progress in those "frontier models" that we have previously seen. There's certainly still gas left in tank tank, but we're way into the diminishing returns by now.
Cloud inference still beats hardware investments by orders of magnitude of course, but that's only if your data doesn't really matter to you.
We are certainly not in the diminishing returns phase for LLM progress. No sign of that yet.
12 replies →
It’s a constant tension in computing that has been around since mainframes and clients… Neither is going to disappear. My general feeling is normal people care more about how thin and light something is than their privacy, so if data center powered LLMs will have a strong future.
1 reply →
they really dont want to hear this bro lol
1 reply →
Or even more amusing in time for the bubble to burst, I hope the big RAM makers end up holding the whole bag for that. Greedy bastards.
Nearly the entire reason we're in this mess is because RAM/SSD/HDD makers are terrified of AI bubble bursting.
Long-term contracts, not building new fabs - they're in full on hedging mode right now.
Since you’re not greedy when there is a memory glut I’m sure you’ll be willing to pay extra to make it fair.
Not at all, I’m going to relax with a cooling drink and watch the consequences of this unbelievably destructive and wasteful venture implode. I also adore the idea that it's on the customer to be "fair" to a company that dumped us a group in favor of chasing B2B money.
Their choice, their consequences.
6 replies →