Comment by mountainriver
1 day ago
Running LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have.
An RTX 6000 pro Blackwell is a pretty good card
1 day ago
Running LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have.
An RTX 6000 pro Blackwell is a pretty good card
A M3 ultra mac Studio can run models that do not fit in similarly priced computers with multiple Nvidia GPUs. And it will use a lot less electricity while still having good enough performance. Except the pre-filing perfs that are quite poor on the M3.
M5 pro 48GB should be good and future proof
If you buy Mac get at least 256GB ram otherwise just buy a bunch of nvidia cards. It really does not make sense otherwise if you are looking for performance / $. The mac (studio) is unique as it has more ram than the alternatives(I.e consumer nvidia cards or spark stuff) so it can fit bigger models but otherwise its performance is worse.