Comment by jnwatson
14 hours ago
M5 Ultra has 4-5x the memory bandwidth of both. 1.2 TB/s memory bandwidth opens up good performance on relatively large models.
14 hours ago
M5 Ultra has 4-5x the memory bandwidth of both. 1.2 TB/s memory bandwidth opens up good performance on relatively large models.
you can get 4xGB10 for <20K so that gets you about the same tg and pp will be probably better. Power consumption though will be something like 200W idle so that's a bummer. And you get VLLM and SGLANG unlike them mac where one has to use oMLX (nice but not the same concurrency or cache performance) I have 128GB M5 Max laptop and sill prefer to run things on other boxes in basement because its no fun to have the primary device being overloaded.
4XGB10 is in quite a different price bracket than a single M5 128GB Mac Studio...