Comment by revolvingthrow
7 hours ago
Already on HuggingFace: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
The bad news is that the original v4 flash was 284B, which was large but still somewhat reasonable for running locally. This one is 552B so almost twice that, so the huge gains in benchmark scores make sense - it's not really flash anymore, imo.
I've no idea about actual performance vs benchmaxxing, though deepseek was fairly trustworthy as far as Chinese models go. If that holds (and if it doesn't think forever, as deepseek 4 sometimes did) it's probably the newest king of the hill amongst open weights models.
It does include vision, and they do something funky with KV cache so it's very efficient: "[...] these designs reduce the global KV cache footprint to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash". I do appreciate the high focus on efficiency, but at this point we sure could use a flash-flash version.
@edit: I couldn't make sense what the actual parameter count is, with the addition of Engram memory. To my understanding the 4.1 flash is 552B parameters you want in vram or ram, out of which ~16B is active (8B for prefill). It also includes additional 196B Engram memory which you can put on an SSD. I think.
Assuming that's correct 256 GB memory is insufficient to even load the model at q4 - you'd be 1GB short, assuming you can fill it to 100% (so no mac). You'd also want some for kv cache of course. A 256 GB desktop with some extra VRAM from GPU could run it, but normal consumer boards get real slow once you fill 4 slots so you'll probably want quad channel which is Threadripper or above territory.
>This one is 552B so almost twice that, so the huge gains in benchmark scores make sense - it's not really flash anymore, imo.
It uses fewer active parameters, though. (8B or 14B instead of always 13B)
So ... flash indeed.
200B of those 552B is PLE, which works more like a database that is read for each token, thus can be offloaded to a fast SSD.
Id love an ELI5 for PLE. Im trying to work it into my back of the napikin math for compute vs memory bandwidth limitations on tok/s in PP vs TG work.
My attempt at a simplification of this article on it https://sebastianraschka.com/llm-architecture-gallery/per-la... into a couple of sentences is that they are linear embeddings of the input token space projected per layer, which are then gated by the transformer outputs per layer.
This would mean that the only one set of weights for the ple path needs to be pumped across the memory bandwidth as they are the same linear weights for all layers?
Sheit, maybe im trying to simplify something that i need to look at in detail. but id love to leverage others understanding if possible
1 reply →
Unfortunately no, it's 200B + 552B. It's not as bad as it sounds though, because most of 552B is in 4bit natively.
1 reply →
[flagged]
it's not really flash anymore, imo.
Flash is about speed ... Flash models are supposed to be fast, way faster then their big brothers that are "better" but way slower.
Its just that up to now, getting more speed involved cutting back on the parameter count, what ended up making the Flash models more "dumber" in exchange for speed.
What we see with DS v4.1 Flash, is that DeepSeek has found a way to make a Flash model, that is 2x a 2.5x faster then the older Flash version, while increasing the intelligence (more parameters). To the point that it goes past Kimi K3 and GLM 5.3 in most tests, with a blazing 250 to 400t/s.
AND its also priced as a Flash model (they even reduced the price back to almost old v4 Flash price), despite it now rivaling those 10x to 30x more expensive competitors.
The issue that people can not fit it into local setups, is not how companies design their models. They design it for their own needs. A old flash needed less parameters to be fast, and local users had the benefit of it fitting in 256GB memory.
Companies who run locally, are perfectly able to buy a few H200/B200 and get a setup that run a model that almost rivals Opus 5.0 in their office. How to say this without getting downvoted. People get way too fired up if a model does not fit, despite that they can still run the old v4.0, qwen 27b, 35b, 3.8 Next and other models. The fact that these models are being released for free, is already amazing by itself. I am still waiting to see what Anthropic and OpenAI and Google are releasing for free... O wait ... ;0
I highly doubt that it goes past Kimi K3 in actual practice. GLM 5.3 claimed the same, but in practice, K3 is so damn knowledgeable and I suspect due to it's massive size.
> Companies who run locally, are perfectly able to buy a few H200/B200 and get a setup that run a model that almost rivals Opus 5.0 in their office.
I agree with your broader point about Flash being about speed not total model size, but I think we should also point out that H200/B200's are seriously overkill for the "run a model in your office" scenario. That sort of hardware is optimized (in a roofline analysis sense) for running hundreds of concurrent sessions on a 24/7 basis. You're severely overpaying for your VRAM in basically any typical local-inference scenario, you should most likely be buying gear based on LPDDR and Flash memory instead which will slash your cost by orders of magnitude.
> It also includes additional 196B Engram memory which you can put on an SSD. I think
You can put Qwen 3.8 Flash Next engram on SSD, but prompt processing takes a good hit. On my mac studio, I get 300 pp and 33 tg with SSD offload, versus 550/40 with everything in RAM.
I will be very happy if 300 pp is achievable with this model though.
The engram stuff is great because RAM is often still cheaper (or at least expandable). My company does currently look into buying some hardware as we handle confidential data and code.
Qwen 3.8 Flash is viable on two Nvidia 6000 96GB with a wood quant because you can put the 50GB Engram into RAM and the hit should be below 10% performance. At least that is what I have seen so far. Correct me if I'm wrong.
You can warm cache regularly used engram/n-gram if you're willing to merge PRs into a personal branch and build it yourself. I was trying this with qwen 3.8 flash next and the n-gram to get it to fit on my very average gaming desktop (it worked)
V4 Flash also was released as mostly FP4, but this one is FP8 (?). 160GB vs 510GB.
Original Flash good fit for dual Spark / Strix Halo machines. This one would require third party quants and even then 4 machines.
Edit: Most of added weights/size are Engrams?
> Overall, DeepSeek-V4.1-Flash has 552B backbone parameters and 196B Engram parameters, activating 8B parameters per token during prefill and 16B during decode.
Those can stay on SSD. So I guess / it possible, that non-engram portion is still FP4 of ~same size! Need to read tech report.
It's larger than previous V4 Flash.
So 384GB needed for a chance of achieving useful speeds. Three Sparks or quad RTX PRO 6000.
Question is how many of those experts one needs to keep in vram for a given workload.
I could imagine (though I might be _very_ wrong there) that for example coding does not live in all of them. Maybe 1/3? Do we have real numbers there?
So maybe one can get away without much performance penalty by doing some LRU stuff?
Or two gorgon halos?
1 reply →
Actually this ought to run quite well with SSD streaming. The MoE expert sparsity seems to be similar to DSv4 Pro (hence exceptionally sparse) but with far fewer total and activated params. The added engram params can reside on disk as well (similar to Qwen Flash-Next), the additional load on storage performance will be quite negligible for typical scenarios.
By reducing per-session KV cache requirements even further compared to DSv4 Flash, this model likely opens up near-frontier model inference (in slow, unattended scenarios) even on low-end consumer hardware, as long as it has enough fast storage to host the model weights. This will be extremely exciting.
it is a way bigger model with extra 200B engram so of course the score improves.
can't wait for deepseek v4.1 pro