← Back to context Comment by maxloh 7 hours ago I thought Q4_K_M is the standard. Why did you choose the 6-bit variant? Does it generate better input? 1 comment maxloh Reply SlavikCA 7 hours ago There is no standard.The higher quantization - the better results, but more memory is needed. Q8 is the best.
SlavikCA 7 hours ago There is no standard.The higher quantization - the better results, but more memory is needed. Q8 is the best.
There is no standard.
The higher quantization - the better results, but more memory is needed. Q8 is the best.