Comment by chpatrick
11 hours ago
But they're pretty fast and can have loads of RAM, which would be prohibitively expensive with Nvidia.
11 hours ago
But they're pretty fast and can have loads of RAM, which would be prohibitively expensive with Nvidia.
A 128GB 2TB Dell Pro Max with Nvidia GB10 is about $4200, a Mac Studio with 128GB RAM and 2TB storage is $4100. So pretty comparable. I think Dell's pricing has been rocked more by the RAM shortage too.
Unfortunately the GB10 is incredibly bandwith starved. You get 128gb ram, but only 270GB/s bandwidth. The M3 Ultra mac studio gets you 820GB/s. (The M4 max is at 410GB/s. I'm not aware of any workload that gets the GB10 to it's theoretical peakflops.
You can't get a 128GB M3 Ultra, it's also more expensive. For some workloads the Studio is better, for others the GB10.
~not unified memory tho~
It is unified memory on this one
4 replies →
I took ~ to be a "singing tone" for some reason till I saw sibling and realized it might be an attempted strikethrough xD
That won't hold much benefit as SOCAMM2 and LPCAMM2 get more popular.
> So pretty comparable.
The Mac Studio almost certainly uses at least half the power
(educated guess, I'm too lazy to go look at all the spec sheets and run the numbers)
It's actually reversed. The GB10 chipset has a TDP of 140w, whereas M2/M3 Ultra pulls over 250w from the wall: https://support.apple.com/en-us/102027
3 replies →
Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.
AFAIK, for the unified bandwidth, it depends mostly on the CPU, for M4 Max (I think it's the default today?) it does ~550 GB/s, while GB10 does ~270 GB/s, so about a 2x difference between the two. For comparison, RTX Pro 6000 does 1.8 TB/s, pretty much the same as what a 5090 does, which is probably the fastest/best GPUs a prosumer reasonable could get.