Comment by aseipp
2 days ago
The new p150 cards linked in the OP have 32GB GDDR6 @ 512GB/s for $1,300. Which isn't bad on paper, I guess. They're meant to be networked (quad 800GB QSFP-DD) like Nvidia GPUs, so two of them would get you 64GB of VRAM at $2600 for ~600W which is basically what you're asking for? The power usage isn't good enough yet at scale I think, but for a workstation it's quite manageable.
Real workloads remain to be seen, but if they can actually get a working build of vLLM and their cards remain actually buyable, well, they're doing better than some of the competition...
> so two of them would get you 64GB of VRAM at $2600 for ~600W which is basically what you're asking for?
Almost, except with respect to space in the box and power usage, which are critical IMHO.
> but if they can actually get a working build of vLLM and their cards remain actually buyable, well, they're doing better than some of the competition...
That's a big if though, poor software support is to be expected and you'll need to factor that in IMHO, and that's why they need to beef up the memory. Of course if software support is stellar then it may be good enough of a deal.