Comment by siquick

This may help you work out the best quant to use for your use case.

https://www.siquick.com/blog/model-quantization-fine-tuning-...