Comment by plagiarist
8 hours ago
Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.
8 hours ago
Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.
AFAIK, for the unified bandwidth, it depends mostly on the CPU, for M4 Max (I think it's the default today?) it does ~550 GB/s, while GB10 does ~270 GB/s, so about a 2x difference between the two. For comparison, RTX Pro 6000 does 1.8 TB/s, pretty much the same as what a 5090 does, which is probably the fastest/best GPUs a prosumer reasonable could get.