Comment by Gareth321
4 hours ago
While true, the news here is the size of the unified RAM. Nvidia only exceeded 256GB RAM in the 2025 B300 - 288GB. The B300 alone (without the baseboard/PSU/chassis/wiring/CPUs/system RAM/etc) is at least 700% more expensive. This enables large language models on consumer hardware. 1200GB/s is plenty for many tasks.
> 1200GB/s is plenty for many tasks.
This + due to the hardware being so prohibitively expensive, we're seeing software optimizations happening. Like that dflash2 stuff for example, or an LRU for MoE and all that kind of stuff.
I wonder if the inference acceleration companies will ever produce a consumer product