Comment by wren6991

DeepSeek Engram paper published: 12th January

Qwen3.8 Flash Next release date: 26th August

DeepSeek V4.1 Flash release date: 10th September

Current date: 6th October

I think they'll become more popular in the coming months. Also Gemma 4 PLE (April) is similar to DeepSeek Engram in a lot of ways, just with 1-grams.

On the proprietary model point: I'm personally curious about whether heavy n-gram offload is one reason Anthropic keep driving down their token vocabulary size (the other reason being eliminating the LM head gradient bottleneck).