Comment by gcanyon
2 months ago
If Apple offered a reasonably-priced laptop with more than 24gb of memory (I'm writing this on a maxed-out Air) I'd agree. I've been buying Apple laptops for a long time, and buying the maximum memory every time. I just checked, and I see that now you can get 32gb. But to get 64gb I think you have to spend $3700 for the MBMax, and 128gb starts at $4500, almost 3x the 32gb Air's price.
And as far as I understand it, an Air with an M3 is perfectly capable of running larger models (albeit slower) if it had the memory.
You’re not wrong that Apple’s memory prices are unpleasant, but also consider the competition - in this context (running LLMs locally) laptops with large amounts of fast memory that can be purposed for the GPU. This limits you to Apple or one specific AMD processor at present.
An HP Zbook with an AMD 395+ and 128Gb of memory apparently lists for $4049 [0]
An ASUS ROG Flow z13 with the same spec sells for $2799 [1] - so cheaper than Apple, but still a high price for a laptop.
[0] https://hothardware.com/reviews/hp-zbook-ultra-g1a-128gb-rev...
[1] https://www.hidevolution.com/asus-rog-flow-z13-gz302ea-xs99-...
Yeah, I'm by no means saying that Apple is uniquely bad here -- it's just an issue I've been frustrated by since the first M1 chip, long before local LLMs made it a serious issue. More memory is always a good idea, and too much is never enough.
You can get any low spec laptop that has no soldered DIMMs and just replace them with the maximum supported capacity.
You don't necessarily need to go the maxed up SKU.
Would that be unified memory? Where the gpu and cpu can share the memory? Which is key for performance.
3 replies →
The framework desktop will get you the 395+ and 128gb of ram for 2k USD.
The trick here is buying used. Especially for something like the m1 series there is tremendous value to be had on high memory models where the memory hasn't changed significantly over generations compared the cpus and even m1's are quite competent for many workloads. Got a m1 max 64gb ram recently for I think $1400.
I think pricing is just one dimension of this discussion — but let's dive into it. I agree it's a lot of money. But what are you comparing this pricing to?
From what I understand, getting a non-Apple solution to the problem of running LLMs in 64GB of VRAM or more has a price tag that is at least double of what you mentioned, and likely has another digit in front if you want to get to 128GB?
it's astonishing how apple gouges on the memory and ssd upgrade prices (I'm on an M1 w/ 64Gb/4Tb).
That said they have some elasticity when it comes to the DRAM shortage.
The M-series unified memory is built into the chip itself, not separate components. Of course Apple is going to maintain their margins, but it’s easy to see why with this design more memory is more expensive than drams. Well maybe not with the current market pricing which hopefully is temporary.
They gouge you on RAM and SSD but provide a far better overall machine for the price than Windows laptops.