← Back to context

Comment by petu

12 hours ago

V4 Flash also was released as mostly FP4, but this one is FP8 (?). 160GB vs 510GB.

Original Flash good fit for dual Spark / Strix Halo machines. This one would require third party quants and even then 4 machines.

Edit: Most of added weights/size are Engrams?

> Overall, DeepSeek-V4.1-Flash has 552B backbone parameters and 196B Engram parameters, activating 8B parameters per token during prefill and 16B during decode.

Those can stay on SSD. So I guess / it possible, that non-engram portion is still FP4 of ~same size! Need to read tech report.

4 comments

petu

Reply
petu  11 hours ago

It's larger than previous V4 Flash.

  552B in ~FP4, 306GB.   
  196B of FP8 Engrams, another 204GB, not necessary to keep in RAM.  
  KV cache sees another 4x size reduction, just 900MB for 1M.

So 384GB needed for a chance of achieving useful speeds. Three Sparks or quad RTX PRO 6000.

  • hypfer  7 hours ago

    Question is how many of those experts one needs to keep in vram for a given workload.

    I could imagine (though I might be _very_ wrong there) that for example coding does not live in all of them. Maybe 1/3? Do we have real numbers there?

    So maybe one can get away without much performance penalty by doing some LRU stuff?