Comment by tgrowazay

5 hours ago

> M5 Pro supports up to 64GB of unified memory with up to 307GB/s of memory bandwidth, while M5 Max supports up to 128GB of unified memory with up to 614GB/s of memory bandwidth

Which roughly translates to 30B Q8 size LLM at 10t/s for the M5 Pro and 60B Q8 size LLM at 10t/s for the M5 Max

For reference, RTX 3090 24GB has a memory bandwidth of approx. 936.2 GB/s, DGX Spark 128GB features a unified memory bandwidth of up to 273 GB/s

0 comments

tgrowazay

Reply