I want 31B. The 12B 4-bit QAT is already small enough to run well enough on every device I use regularly, including phone and tablet, I don't need a 1-bit or ternary version of the 12B.
But, what I really want is for Google to release bigger Gemma 4 models, particularly a bigger MoE, like a ~70B or ~120B. Gemma 4 is the best all-rounder among the models I can self-host even though I've got a 128GB Strix Halo. A 4-bit QAT version of a 70B MoE would probably be the sweet spot.
A bigger Qwen 3.6 with a 4-bit QAT version would also be welcome, as the prior bigger versions aren't notably better than 3.6 27B, but I guess Qwen is done doing larger open weights models. They did release AgentWorld recently, a post-train of the 3.6 MoE, so they're still doing some open things.
I think I want to see more third-party testing of this ternary Qwen to know if crushing it to 1.56 bits kills it; there are tons of benchmarks of Qwen 3.6 27B, so it's an ideal candidate to figure out what the extreme compression does to it.
I want 31B. The 12B 4-bit QAT is already small enough to run well enough on every device I use regularly, including phone and tablet, I don't need a 1-bit or ternary version of the 12B.
But, what I really want is for Google to release bigger Gemma 4 models, particularly a bigger MoE, like a ~70B or ~120B. Gemma 4 is the best all-rounder among the models I can self-host even though I've got a 128GB Strix Halo. A 4-bit QAT version of a 70B MoE would probably be the sweet spot.
A bigger Qwen 3.6 with a 4-bit QAT version would also be welcome, as the prior bigger versions aren't notably better than 3.6 27B, but I guess Qwen is done doing larger open weights models. They did release AgentWorld recently, a post-train of the 3.6 MoE, so they're still doing some open things.
I think I want to see more third-party testing of this ternary Qwen to know if crushing it to 1.56 bits kills it; there are tons of benchmarks of Qwen 3.6 27B, so it's an ideal candidate to figure out what the extreme compression does to it.