← Back to context

Comment by jmyeet

8 hours ago

The selling point of the M5 Ultra Mac Studio is that you can run much larger models that the 5090 can't without swapping. NVidia aggressively segments the market on VRAM for this reason. That's why a 5090 has an MSRP of ~$2k (but good luck getting one for less than $4k) while a 6000 Pro, which is basically a 5090 with 96GB of RAM has now soared beyond $15k where 3-6 months ago it was more like $10-11k. A 6000 Pro has the same memory bandwidth but slightly more CUDA units (IIRC ~24k vs ~21k).

This advantage won't be apparent with a 27B model. The 256GB MS can probably run the newer Flash models locally, something you can't do on a 5090.

I don't think we'll get a successor to the 5090 until late 2028, maybe even 2029. I'm basing this on the launch date of the 5000 series and that we haven't got a midcycle refresh yet. Rumor has it the chips are ready but the 3GB RAM modules are 3-4x the price of the 2GB modules used on the current cards.

Apple should see a Mac Studio major update in 2028. That might even force NVidia's hand. But it's really impossible to say what the state of the market will be 2-3 years from now. It may have completely crashed. I suspect not however.

The interesting thing will be when the bandwidth demands start forcing HBM memory onto these home/enthusiast solutions.

11 comments

jmyeet

Reply
pama  8 hours ago

But what about builds that combine 8 of the 5090 with infiniband between boxes? Wouldn't that be comparable to the mac in terms of price and potentially beat it by a lot in terms of performance for the large MoE? I understand the space/heat/noise considerations, but price wise it may still not make as much sense as people think. (Agreed that it is hard to get the NVIDIA hardware and the 6000 pro are priced less competitively).

  • happyopossum  2 hours ago

    >8 of the 5090

    Where are you buying 8 5090s for under $10k? With CPU, RAM, and (checks comment) infiniband hardware???

    You're probably looking at a lot closer to $60k when all is said and done, and that's before you hire an electrician to run a sub panel for your homelab...

  • glitchc  4 hours ago

    Others have chimed in on cost and size, I'll chime in on power. The 8x 5090s will require a dedicated datacentre grade power source. The Mac Studio runs on a plain jane wall socket.

  • kridsdale1  8 hours ago

    While that sounds super awesome, How many people are actually going to build and maintain that vs a box you can grab at the mall that fits in a lunchbox?

  • jmyeet  7 hours ago

    I can't speak to Infiniband pricing for something like that. It seems like the cheap option is 56/100Gbps with used Enterprise equipment. You'd need 8 HCAs, DAC cabling and a switch but even then you're into thousands of dollars. If you want 200Gbps+ it gets into the tens of thousands (AFAICT).

    Each PC is probably going to cost ~$6k and you're talking about 8000W of electricity draw. That's going to consume multiple 20A circuits even at 240V. And the electricity ain't free either. A Mac Studio seems to draw ~500W max.

    Oh and the Mac Studio has an upgrade route to run 1T+ models too by chaining them together with TB5 chaining. OSX supports RDMA this way. That's comparable bandwidth to the 100Gbps Infiniband option.

    So you're talking about $50-60k of hardware and more power draw and more heat for something that will I'm sure beat the MS M5U option but at huge cost. Also, at that kind of price point, I'm likely to get a workstation PC and put 2 (or possibly 3) 6000 Pros in it.

weee322  4 hours ago

openai make a npu google make npu (tpu no mater) amd buy tellas

every company make his own npu (without xai)

probaby in 2028 we will have more concurent firm on market place