Comment by doctorpangloss 8 hours ago the problem is the dsv4 checkpoint so quantized isn't very good 4 comments doctorpangloss Reply ilaksh 5 hours ago Which ds4 checkpoint for which model exactly did you test? Don't they have multiple different versions and quantization levels? c0rruptbytes 5 hours ago not my experiencethe ds4 quants were very good beating the unsloth quants https://github.com/michaelasper/benchmarks/blob/main/deepsee... dotancohen 5 hours ago That's quite the statement - unsloth quants are amazing.
ilaksh 5 hours ago Which ds4 checkpoint for which model exactly did you test? Don't they have multiple different versions and quantization levels?
c0rruptbytes 5 hours ago not my experiencethe ds4 quants were very good beating the unsloth quants https://github.com/michaelasper/benchmarks/blob/main/deepsee... dotancohen 5 hours ago That's quite the statement - unsloth quants are amazing.
Which ds4 checkpoint for which model exactly did you test? Don't they have multiple different versions and quantization levels?
not my experience
the ds4 quants were very good beating the unsloth quants https://github.com/michaelasper/benchmarks/blob/main/deepsee...
That's quite the statement - unsloth quants are amazing.