Comment by SlavikCA

4 hours ago

I'm getting 30 t/s on RTX 4090D (using 42 out of 48GB VRAM) with UD-Q6_K_XL

2 comments

SlavikCA

I thought Q4_K_M is the standard. Why did you choose the 6-bit variant? Does it generate better input?

SlavikCA 4 hours ago

There is no standard.
The higher quantization - the better results, but more memory is needed. Q8 is the best.