Comment by yfontana
8 hours ago
How come no other big model seems to be able to deliver the same type of extremely low cache cost though, if their techniques are public?
8 hours ago
How come no other big model seems to be able to deliver the same type of extremely low cache cost though, if their techniques are public?
They can and almost certainly are doing similarly impressive engineering works internally.
They just aren't in any hurry to forward those cost savings to you.
I think the "architectural features" are part of the model, not the kv cache. So implementing it would be difficult and expensive.
Deepseek V4 paper is just ~three months old
Many of these techniques haven't been published very long ago - it often takes a good 6-8 months for techniques to percolate. But also, they come at a complexity cost and, seemingly, also at a stability cost.
Also potentially a performance (in terms of output quality) cost. DeepSeek is cheap on a per token basis but lags behind in the benchmarks, perhaps it was a calculated tradeoff.