← Back to context Comment by Tepix 6 hours ago Sounds good. I saw that you use the FP8 version of the model. Do you also quantize the KV cache? 1 comment Tepix Reply sacrelege 4 hours ago no I don't, since there seem to be a silent degradation bug
no I don't, since there seem to be a silent degradation bug