Comment by stared
1 day ago
I wrote this blog post myself, with AI for proofreading (typos and grammar, but not style). There were a few singular sentences for which I had a writer's block, but not much besides that.
So, if there are irrelevant remarks, these are mine. :)
Charts are vibe-coded - but it took quite a bit of hand-holding to get something decent. And the logarithmic scale for model size is my conscious choice (against Claude's initial ideas).
Hi,
If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
If you like, you can run these tests yourself as well. It is ~$500 per a single combination, and assuming no failed runs.