Comment by bigyabai
1 day ago
Both this GPU and the Mac are constrained by a lack of GPU compute. There's basically no rack-scale GPUs with 480gb on-die that have a sub-400W compute element. It's going to be incredibly prefill-constrained on denser model inference just like the Apple Silicon iGPUs are.
This GPU has 1.3 PFLOPS of FP8 compute performance and 1.5 TB/s of memory bandwidth: https://videocardz.com/newz/intel-gpu-d270-is-xe3p-crescent-...
This is almost twice better than RTX 5090. This GPU will not be prefill constrained, I feel.
But I am surprised that they were able to pack so much flops into 300W.
> But I am surprised that they were able to pack so much flops into 300W.
I'm not. When you're doing inference you typically reduce power limits anyway if you're optimizing for efficiency. Even if you don't, it doesn't spike to the max of the card.
My 3090 I power it down to 175W or so for single-digit reductions in performance for like, 60% of the wattage, something like that.
It seems like there’s a market out there that is just happy to run the models locally, even if generation is slow.
Am I wrong to assume that large MoE models are likely the point anyway? Seems like this card would love DeepSeek V4.1 Flash.