Comment by petu
11 hours ago
It's not 1 bit. It's ~4bit for n-gram and ~2.8bit for the model. Not idea why it's called Q1, but likely it's preliminary quant just for PR testing / very likely to be remade after llama.cpp support is merged.
11 hours ago
It's not 1 bit. It's ~4bit for n-gram and ~2.8bit for the model. Not idea why it's called Q1, but likely it's preliminary quant just for PR testing / very likely to be remade after llama.cpp support is merged.
No comments yet
Contribute on Hacker News ↗