Comment by genpfault
7 hours ago
600 GB/s of memory bandwidth isn't anything to sneeze at.
~$1000 for the Pro B70, if Microcenter is to be believed:
https://www.microcenter.com/product/709007/intel-arc-pro-b70...
https://www.microcenter.com/product/708790/asrock-intel-arc-...
Recent kernels have SR-IOV support for these chips too. B&H has them listed for $950.
https://www.bhphotovideo.com/c/product/1959142-REG/intel_33p...
When 32GB NVIDIA cards seem to start at around $4000 that's a big enough gap to be motivating for a bunch of applications.
I'm probably going to snag one of the Intel cards just for the SR-IOV and use with VM's
I think the B65 is priced at $650. Both supported by llamacpp I believe. With that power draw you could run two of them.
Intel GPU prices have stayed fine, but I do wonder if they are viable for Inference if they will wind up like Nvidia GPUs, severely overpriced.
I mean it kind of is considering that's comparable to a 5070 which has 672 GB/s? Benefit of NVIDIA being the only one using GDDR7 for now I guess.
7800 XT has 624 GB/s as well, and can be found for $400 used. 16 GB of course.
I tend to agree that the vram size and bandwidth is the core thing, but this B70 Pro allegedly has 387 int8 tops vs a 5090 having 3400 int8 tops. 600 compares vs 1792GB/s. I'm delighted so see an option with quarter the price! But man, a tenth the performance? https://www.techpowerup.com/347721/sparkle-announces-intel-a... https://www.tomshardware.com/pc-components/gpus/nvidia-annou...
838 seems to be the real INT8 TOPS number for the 5090; going from 800 to 3400 takes an x2 speedup for sparsity (so skipping ops) and another x2 speedup for FP4 over INT8.
So it's closer to half the speed than a tenth. Intel also seems to be positioning this card against the RTX PRO 4000 Blackwell, not the 5090, and that one gets more like 300 INT8 TOPS. It also has less memory but at a slightly higher bandwidth. The 5090 is much faster and IIRC priced similarly to the PRO 4000, but is also decidedly a consumer product which, especially for Nvidia, comes with limitations (e.g. no server-friendly form factor cards available, and there are or used to be driver license restrictions that prevented using a consumer card in a data center setup).
Thank you for the correction. That seemed way too lopsided to be believed. This assessment balances the memory to tops ratio much much more evenly, which is to be expected! I was low key hoping someone would help me make sense of how wildly disparate figures were, but I wasn't seeing.
AMD R9700 is 378/766 tops int8 dense/sparse. 644GB/s of 32GB memory. ~$1400. To throw one more card into the mix. Intel undercutting that nicely here.
You're right that for companies, the pro grade matters. For us mere mortals, much less so. Features like sr-iov however are just fantastic so see! Good job Intel. AMD has been trickling out such capabilities for a decade (cards fused for "MxGPU" capability) & it makes it such an easier buy to just offer it straight up across the models.
especially for exploratory work 1/10th the perf is fine. Intel isn't able to compete head to head with Nvidia (yet), but vram is capability while speed is capacity. There will be plenty of use cases where the value prop here makes sense.
It's more like a 70 class card with extra VRAM.
The product would be excellent in 2024, but now it's a landfill filler. You can run some small models at pedestrian speed, novelty wears off and that's it.
Intel is not looking in the future. If they released Arc Pro B70 with 512GB base RAM, now that could be interesting.
32GB? Meh.