Comment by plasticchris
2 days ago
Qwen’s 3.6/3.8 27b actually has some algorithmic advantage when it comes to the kv cache size needed, so it actually needs less memory at equivalent context. My experience using both in vllm supports, with considerably more overhead in context size on these models than Gemma 4 31b, and better performance in most tasks I’ve tried on both models.
Qwen 3.x does have an advantage but it's relatively small (64KB/token vs 80KB/token) - Gemma4 actually has less % of full attention layers, but the largest geometry and has the biggest "fixed" state for it's non-global layers. Muse Glimmer actually has by far the lowest per-token cache usage for the competitive 30B-class dense models - it's at about 13KB/token - very aggressive GQA (32Q/2KV) and also by far the smallest QKV dimensions.
Actually perf (speed) is going to mostly on token output, and here Qwen 3.x historically tends to lose badly as it tends to overthink a lot. I'll be running evals on 3.8 myself this weekend to see how its reasoning levels perform.
I assume that AA will have 3.8 numbers soon and Intelligence Index vs Output Tokesn per Intelligence Index Task is a decent way to view that: https://artificialanalysis.ai/models/muse-glimmer?intelligen...