Comment by thehamkercat
5 hours ago
For local LLM people:
(for comparison, M4 pro has 273 GB/s and M1 pro has 200GB/s and M1 max has 400GB/s)
M5 Pro (20-core GPU): 307 GB/s memory bandwidth (supports 24 GB, 48 GB, 64 GB).
M5 Max (32-core GPU): 460 GB/s memory bandwidth. (36 GB)
M5 Max (40-core GPU): 614 GB/s memory bandwidth (supports 48 GB, 64 GB, 128 GB)
IMO getting a studio for LLMs is a wiser choice since it doesn't heat up and there are no batteries to degrade.
For an M5 max top bin with 128GB Ram:
- mac studio: $5,899.00 (with 2TB ssd bump to match the default macbook sku)
- macbook 16: $7,399.00
Portability gets a hit but I feel you can have a thin client and let the studio do the heavy lifting back at your home/studio(heh).
I have a limited understanding of local LLMs, but from what I gather 128GB is not enough to match DeepSeek v4 Flash or GPT 6.1 Luna. And these models are so cheap right now. What's the use case besides having your data locally?
"Having your data locally" is definitely the biggest one.
- The general consensus seems to be that the frontier models are kind of (understandably) "benchmaxxed" towards coding benchmarks and have suffered in some other areas like writing, art, etc. Lot of tweaked/specialized models on huggingface
- Getting around the censorship in commercial models
- While local models don't match frontier coding agents, you can run them for multiple 24/7 with a long running goal (The fans in Macs are presumably not built to run 24/7, but this can be mitigated to a considerable extent IME by simply pointing a desktop fan at it, and you can throttle things in software. On the discrete GPU side of things, you can usually get big power savings without much performance impact by throttling the GPU clock; things are usually bottlenecked on memory)
- Even small 27B/35B models are getting darn good, definitely more than powerful enough for classification etc... you don't need 128GB to run them
- Right now, providers like OpenAI are running at huge losses. At some point, they are going to need to turn a profit, at which point they may get significantly more expensive, and "hosting your own" may make more economic sense than it does today -- and be a valuable professional skill, even.
- Even more speculative: perhaps the future for many applications is training and running your own hyper-specialized models. Right now, training your own is nowhere near cost effective on self-hosted hardware vs. renting GPUs
Note: whether any of these use cases are "worth it" is up to the individual. I'm not trying to sell anybody on the idea. I don't personally do much local LLM stuff at the moment, but it's attractive and I've been close to pulling the trigger...
2 replies →
While personally I’m not using them for work, just tinkering, 128GB can run qwen 3.8 flash next at q4(~111GB gguf). That model seems to be quite capable from what I’ve read.
It’s quite evident that the api models can and have been nerfed at the provider’s discretion, then there’s the abliteration aspect too(api models refusing what they consider “harmful”) which makes the case for local.
Privacy, mainly. I don't see how it would make economic sense. $10k will buy an awful lot of GLM-5.3 tokens.
Being able to run your own uncensored model the way you want to.
I was on the market for a 128Gb M5 Max and went with the Studio + an Air for about the same money
which configs did you wind up going with on the Studio + Air, I'm in a similar boat. Grabbed a 13" 24GB M5 air before the price hike and now wrestling between a M5 Max studio or an overpriced mini.
These are right at the edge of desirable for me right now -- but it's hard to pull the trigger. I'm imagining an M7 Pro with 480 GB/s and therefore an M7 Max 850-900 GB/s. And significantly better GPU speeds as well. If I knew I could get the M7 Max Q2 of next year, especially at 192GB of RAM, I'd wait.
But then you have to compare to buying a GB200, sticking it in a closet and sharing it out in a workgroup and having a lower specced laptop. Decisions..
Notably these chips came out back in March of this year. Today's changes are in the display and form factor but maintaining the same CPU, GPU, and memory specs.
Rumor is that Apple is skipping the M6 Pro and Max chips this year in favor of releasing the M7 Pro and Max sometime next year.
What's are the odds that the M7 Max goes above 128GB? (Presuming the rumors about Apple skipping the M6 Pro/Max are true)
At this point it's pretty certain they've skipped M6 Pro/Max or they'd be a flagship feature of today's laptop.
The odds of > 128GB are very good, AMD have already bumped the Strix Halo to 192GB of RAM and both AMD and Nvidia should have successors to their offerings in 2028 that will compete with the M7 Pro and Max. Both AMD and Nvidia will be using TSMC N2 too so the competition will be ramping up.
Very high