Comment by jbellis
3 hours ago
Isn't the "KV Compression Strategies (FAIR)" chart showing that the fancy complex algorithm only barely beats simple topk?
The commentary says that topk "degrades rapidly at low ratios" but the same can be seen for HAE (Entropy + OLS).
Is it really that fancy and complex, though? The “entropy recycling bin” seems fancy to me, but the other stuff is least squares and an SVD, these are solid workhorse numerical routines.
Fair point, the gap isn’t huge in that plot, and both degrade at low ratios. The difference is more in how they degrade: TopK can have sharper, localized failures, while HAE tends to be a bit more smooth. That doesn’t always show up strongly in average MSE.
That said, the gains are modest right now, this is still a research prototype exploring the tradeoff, and there’s clearly more work to be done.