Comment by hypfer

6 hours ago

> 1200GB/s is plenty for many tasks.

This + due to the hardware being so prohibitively expensive, we're seeing software optimizations happening. Like that dflash2 stuff for example, or an LRU for MoE and all that kind of stuff.