Comment by CamperBob2
1 day ago
How do you use the RTX 6000 with the Macs? Exo? I would think that would be pretty snappy if configured properly.
1 day ago
How do you use the RTX 6000 with the Macs? Exo? I would think that would be pretty snappy if configured properly.
This is on a separate Windows PC, I don't have it integrated with the Macs.
If you don't need cash right away, I'd wait until the M5 Ultra comes out and see how things shape up. There have been some early efforts aimed at combining the prefill performance of a GPU with the high throughput achievable with the Mac's unified memory architecture (see various YouTube videos by Ziskind and others, as well as https://old.reddit.com/r/LocalLLM/comments/1r6drpi/exo_clust... ).
Point being, once the M5 Ultra is available, I suspect a lot of people will get very serious about making Macs work with RTX GPUs because that will yield an inference platform with a good bang:buck ratio. If so, you may find that your existing hardware is more powerful than it seems today. And it may be a lot more expensive to replace later if you sell it now.