Comment by 0x457

1.76 bpw number is kinda misleading if you compare it directly to IQ2/Q2. The encoding is ternary, but the quantization procedure is way more sophisticated than "round Qwen weights to {-1,0,+1}."

They rotate the weights into a quantization-friendly basis first, then ternarize with per-group scales and error compensation.