← Back to context

Comment by lostmsu

1 day ago

Looking at their benchmarks there doesn't appear to be meaningful difference between their quants and bartowsky quants.

3 comments

lostmsu

Reply
danielhanchen  21 hours ago

No our Qwen3.5 new ones show the opposite see https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks

  • lostmsu  15 hours ago

    Am I misreading the table?

      Unsloth Q4_K_M

  PPL:       6.6053     KLD 99.9%: 0.5478     KLD mean: 0.0192

  bartowski Qwen_Q4_K_M

  PPL:       6.6097     KLD 99.9%: 0.5771     KLD mean: 0.0182

    Barely noticeable drop in PPL; noticeable KLD drop (good, 5%); but worse KLD mean (bad, 5%).

    • danielhanchen  5 hours ago

      You forgot to check the disk sapce - _M and _XL are not the same across quants:

      Unsloth Q4_K_M 18.49GB 0.5478 KLD 99.9% 0.0192 mean

      Unsloth Q4_K_XL 19.17GB 0.4097 KLD 99.9% 0.0137 mean

      bartowski Q4_K_M 19.77GB 0.5771 KLD 99.9% 0.0182 mean