Comment by zozbot234

Actually this ought to run quite well with SSD streaming. The MoE expert sparsity seems to be similar to DSv4 Pro (hence exceptionally sparse) but with far fewer total and activated params. The added engram params can reside on disk as well (similar to Qwen Flash-Next), the additional load on storage performance will be quite negligible for typical scenarios.

By reducing per-session KV cache requirements even further compared to DSv4 Flash, this model likely opens up near-frontier model inference (in slow, unattended scenarios) even on low-end consumer hardware, as long as it has enough fast storage to host the model weights. This will be extremely exciting.