I've seen a few articles from providers talking about KV cache quantisation, but it's not something they explicitly point out like they do with weights.
So you could end up paying more for unquantised weights, only to get silently hit with a quantised KV cache...
I've seen a few articles from providers talking about KV cache quantisation, but it's not something they explicitly point out like they do with weights.
So you could end up paying more for unquantised weights, only to get silently hit with a quantised KV cache...