Comment by ehtbanton
18 hours ago
This is genuinely very helpful. I'm planning a MacBook pro purchase with local inference in mind and now see I'll have to aim for a slightly higher memory option because the Gemma A4 26B MoE is not all that!
18 hours ago
This is genuinely very helpful. I'm planning a MacBook pro purchase with local inference in mind and now see I'll have to aim for a slightly higher memory option because the Gemma A4 26B MoE is not all that!
pretty sure Nvidia GPU is better bang for buck because of usable inference speed..
I have upgraded my M4 Pro 24GB to M5 Pro 48GB yesterday. The same Gemma 4 MoE model (4bit, don't remember which version) runs about 8x faster on M5 Pro and loads 2x times faster in memory.
So yes, do purchase that new MacBook Pro.
You don't know if it's the newer model or the increase in RAM. If someone has already got 48GB it they might not benefit much. You changed 2 things at once.
Not really: it's the same model size and it fits 24GB entirely.
If you're doing it specifically for inference (or in most other situations) a Mac(book) represents very low RoE.
s/RoE/RoI