Comment by Zenbit_UX
10 hours ago
Top spec build is $29,799 + tx. While local AI is certainly an enticing prospect I can’t really imagine myself ever seeing a return on that investment.
Further they’ve already begun hyping the M7 Ultra and even if that got delayed until 2028, this machine only being SOTA for 1.5yrs is a hilariously short window in which to maximize your investment.
Enthusiasts generally don't see a "return on investment" on their luxury cars or their boats either, but they enjoy what they're able to do with them and aren't exactly embarassed about the class signaling of being a high spender.
Add to that the smattering people with genuinely justifiable use cases, and you have more than enough demand for a limit-pushing, highest-margin model of your product platform.
> Enthusiasts generally don't see a "return on investment" on their luxury cars
I hate to break it to you but the same thing has happened with cars. Many in-demand performance cars like the GT3 RS have been appreciating.
I'd be willing to spend that much if it unlocked the ability to run top-of-the-line open weight models at their best quantization at a decent enough speed for personal agentic use. We're clearly not there yet, and I just can't imagine dropping the price of a brand new car on something that will only get me heavily quantized versions of the smaller models.
[flagged]
It is not exactly an investment, but I guess a lot depends on what you actually would want to do with it.
> It is not exactly an investment
At $30,000 it's sure an expensive hobby. You could get into Caterham racing for that...
Those are corporate dollars making that spend, thats the only people.
Wait, where is Apple hyping the M7 Ultra?
They're not. It was a bit of spurious guesswork by Mark Gurman, and we all know how those guesses end up.
The 256GB M5 Ultra Mac Studio can (AFAIK) run the latest open weight Flash models but only 2 bit quant. This by extension will be able to do 4 and that's a big difference. We're not at the point of running 1T+ parameter models locally yet but we'll be there by 2030 easily.
I personally believe that local LLMs have a huge captive market for people who cannot or should not host that data with a third-party. For example, it's an open question of whether or not using cloud LLMs by lawyers breaks attorney-client privilege. HIPAA with medical records is another one.
But I want to expound on one point that you made, which is really important. You say that this spec only has a SOTA life of ~1.5 years and that's an issue. Take that to its natural conclusion. Why isn't it an issue for hyperscalar data centers, particularly as NVidia wants to move to annual chip updates.
I believe the hardware is going to be commoditized. At that point the only thing that matters is performance-per-Watt. NVidia's R100/200 doesn't make the B100/200 obsolete but if the R series gets 50-100% PPW more and these DCs are using gigawatts of power, that's a massive difference.
What really is the effective life of AI hardware?
> Take that to its natural conclusion.
What on earth is a "natural conclusion" in this context? I'm going to assume that part of your comment was written by AI.
> Why isn't it an issue for hyperscalar data centers, particularly as NVidia wants to move to annual chip updates.
Because he's not a hyperscaler? He's not operating data centers where compute is fungible, he's buying an individual machine that he wants to use for years. He can't just keep buying a new machine and fob off the old one to someone else's less demanding workload.
Enterprise GPUs have a lot longer effective life than these consumer chips (although even the old 3090s are reasonably useful, especially with the recent driver unlock tech). I'd argue that these consumer chips have only now gotten to the memory capacity/processing capacity to make local LLM use actually reasonable.