Comment by zozbot234

The new DeepSeek models address this issue very cleanly. DeepSeek Flash V4.1 requires less than 1 GB memory for a full 1M context, down from about ~10 GB in DeepSeek Flash V4.0. This is a significant step towards making near-frontier models usable even on low-end consumer hardware, though of course with significant tradeoffs in overall performance.