Comment by InsidiousWombat
1 day ago
I wonder when we will start seeing small commercial accelerators integrated with NVMes; could potentially use far more bandwidth than typical NVMe drive can push thru the four lanes.
Given that you'd have probably a lot more capacity just to get to required bandwidth it could be used to switch very quickly between models (as all stored there would essentially be "loaded")
AMD Radeon Pro SSG, launched ten years ago at SIGGRAPH: https://www.tomshardware.com/news/amd-radeon-ssg-1tb-gpu-mem...
When High Bandwidth Flash comes out, most likely: https://chipsandcheese.com/p/hot-chips-2026-applying-high-ba...
Seems like a useful innovation on the desktop level, but on the server level, isn't it already pretty normal to see machines with 24x high-performance (i.e. individually high-bandwidth) NVMe drives in them, that can then be used in software RAID0 to achieve pretty massive read bandwidth?
Well, it's not my field, but pci5.0 x4 drives are the fastest I found with a quick Google. So 24 of those would top out at ~400 GB/s. That's also where HBF starts out, for one device. The higher grades are faster, but maybe NVMe will be as fast by the time they are out.
But the win here on servers is almost certainly power. High speed signals are power hungry, and the power scales with the length of the wire. HBF is designed to go in the same package as the GPU, so the distance is much shorter than with discrete drives.
I think what we're going to see first is essentially gpus with nvme (or at least flash) bolted on, not sure if that's also what you're saying or not.