Comment by kennywinker
6 hours ago
I mean, I get pretty damn good general reasoning out of Qwen3.8-27B quantized to 4bits, and that knows Swift, Python, Node, Ruby, Rust, etc. etc. etc.
And so far even the biggest model doesn't seem to have a working Make No Mistakes module, so maybe that's not needed
Qwen3.6-27B (not 3.8) was a much better coder than Gemma4-31B, yet Gemma4-31B was a much better reasoner and general LLM to talk with.
Sure, 3.8 maybe it's better now, but an accurate comparison would be with a new Gemma4-31B iteration (that doesn't exist).
I guess the question is how much of the coding skill gap between qwen3.6-27b and gemma4-31b is down to the model architecture, and how much is because of the training data.
Assuming you can tweak the training data, regenerate qwen3.6, and get a better coder, then presumably you could have variants - e.g. qwen3.6-swift-27b and qwen3.6-python-27b. Or maybe all coding is too intertwined and you can only get splits like qwen3.6-research-27b and qwen3.6-coding-27b. Which isn't quite my pluggable-models dream, but it's a step closer.
But maybe the difference isn't the training data, it's the architecture, in which case pluggable models is probably not possible.