Comment by pama
8 hours ago
But what about builds that combine 8 of the 5090 with infiniband between boxes? Wouldn't that be comparable to the mac in terms of price and potentially beat it by a lot in terms of performance for the large MoE? I understand the space/heat/noise considerations, but price wise it may still not make as much sense as people think. (Agreed that it is hard to get the NVIDIA hardware and the 6000 pro are priced less competitively).
> But what about builds that combine 8 of the 5090 with infiniband between boxes?
Why Infiniband ("IB")? If it's for RDMA, that is possible with certain Ethernet cards/chipsets as well. Certainly Mellanox, but Broadcom:
* https://techdocs.broadcom.com/us/en/storage-and-ethernet-con...
and Intel as well:
* https://www.intel.com/content/www/us/en/support/articles/000...
Link level flow control or priority flow control needs to be supported on the switch ports as well.
No, $40K is not comparable to $10K.
When I compare those two numbers, it seems there's $30k of difference
>8 of the 5090
Where are you buying 8 5090s for under $10k? With CPU, RAM, and (checks comment) infiniband hardware???
You're probably looking at a lot closer to $60k when all is said and done, and that's before you hire an electrician to run a sub panel for your homelab...
Others have chimed in on cost and size, I'll chime in on power. The 8x 5090s will require a dedicated datacentre grade power source. The Mac Studio runs on a plain jane wall socket.
While that sounds super awesome, How many people are actually going to build and maintain that vs a box you can grab at the mall that fits in a lunchbox?
Sounds like nice utility bill in the making.
I can't speak to Infiniband pricing for something like that. It seems like the cheap option is 56/100Gbps with used Enterprise equipment. You'd need 8 HCAs, DAC cabling and a switch but even then you're into thousands of dollars. If you want 200Gbps+ it gets into the tens of thousands (AFAICT).
Each PC is probably going to cost ~$6k and you're talking about 8000W of electricity draw. That's going to consume multiple 20A circuits even at 240V. And the electricity ain't free either. A Mac Studio seems to draw ~500W max.
Oh and the Mac Studio has an upgrade route to run 1T+ models too by chaining them together with TB5 chaining. OSX supports RDMA this way. That's comparable bandwidth to the 100Gbps Infiniband option.
So you're talking about $50-60k of hardware and more power draw and more heat for something that will I'm sure beat the MS M5U option but at huge cost. Also, at that kind of price point, I'm likely to get a workstation PC and put 2 (or possibly 3) 6000 Pros in it.