Comment by cyanydeez
7 hours ago
for comparison, Qwen3.8-Flash-Next only requires 6B parameters for computation, but stores 125B, 51B of those can be comfortably offloaded as they're not actively used in decode but a single token look up.
The Quant iQ4 of this model loads, then, in ~60GB of vram, and on disk it's 85GB.
So if you could etch it, you'd need a ~25GB ssd chip and 60GB of vram.
The vram costs likely contributed to these things being out of reach of the current economic cycle.
You don't want to use a sparse model for a Taalas-like design. Something like a Qwen 3.8 27B makes much more sense.
I'm not smart enough to know why; I do know that 27B is greater for short/interactive on blackwell, but the intellgence leap of the MoE in Qwen3.8-Flash-Next is quite remarkable.
I'm pretty convinced the pathway to local models will be MoE, especially if they can find a way to keep tweasing out things like PLE into the slow bandwidth lanes.