Comment by Ey7NFZ3P0nzAe
6 hours ago
Hi,
If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
6 hours ago
Hi,
If you ever get to writing a blog post about kv cache quantisation, i'm interested in quantising K differently than V
If you like, you can run these tests yourself as well. It is ~$500 per a single combination, and assuming no failed runs.