Comment by benjiro29
4 hours ago
it's not really flash anymore, imo.
Flash is about speed ... Flash models are supposed to be fast, way faster then their big brothers that are "better" but way slower.
Its just that up to now, getting more speed involved cutting back on the parameter count, what ended up making the Flash models more "dumber" in exchange for speed.
What we see with DS v4.1 Flash, is that DeepSeek has found a way to make a Flash model, that is 2x a 2.5x faster then the older Flash version, while increasing the intelligence (more parameters). To the point that it goes past Kimi K3 and GLM 5.3 in most tests, with a blazing 250 to 400t/s.
AND its also priced as a Flash model (they even reduced the price back to almost old v4 Flash price), despite it now rivaling those 10x to 30x more expensive competitors.
The issue that people can not fit it into local setups, is not how companies design their models. They design it for their own needs. A old flash needed less parameters to be fast, and local users had the benefit of it fitting in 256GB memory.
Companies who run locally, are perfectly able to buy a few H200/B200 and get a setup that run a model that almost rivals Opus 5.0 in their office. How to say this without getting downvoted. People get way too fired up if a model does not fit, despite that they can still run the old v4.0, qwen 27b, 35b, 3.8 Next and other models. The fact that these models are being released for free, is already amazing by itself. I am still waiting to see what Anthropic and OpenAI and Google are releasing for free... O wait ... ;0
> Companies who run locally, are perfectly able to buy a few H200/B200 and get a setup that run a model that almost rivals Opus 5.0 in their office.
I agree with your broader point about Flash being about speed not total model size, but I think we should also point out that H200/B200's are seriously overkill for the "run a model in your office" scenario. That sort of hardware is optimized (in a roofline analysis sense) for running hundreds of concurrent sessions on a 24/7 basis. You're severely overpaying for your VRAM in basically any typical local-inference scenario, you should most likely be buying gear based on LPDDR and Flash memory instead which will slash your cost by orders of magnitude.
> I think we should also point out that H200/B200's are seriously overkill for
I simply mention what came to mind ;)
A quad 6000 with 96GB, can run this model at NVFP4. That is 60.000 Euro for the GPUs and lets be generous with another 20.000 for the rest of the system. The price of a single developer for a year.
I highly doubt that it goes past Kimi K3 in actual practice. GLM 5.3 claimed the same, but in practice, K3 is so damn knowledgeable and I suspect due to it's massive size.