Comment by seanmcdirmid
12 hours ago
That MoE model is fast. 30B A3B gets 140 tokens/second on my M3 Max, and…the A3B part means a reasonable KV cache size and reasonable prefill times. 3.8 27B is a dense model that is mostly unusable, 40 tokens/second using MTP sure, but any cache miss is a few minutes of prefill. You are better off going with a lobotomized 3.8 flash MoE for the same amount of memory.
The report is still bizarre, but that model is still very relevant to local users.
I have no idea why you wouldn't be using Qwen3.6 35B-A3B with that setup. Or did you mistakenly think that was what they're talking about?
Their model is over a year old, like, a few decades in LLM history.
Or even Qwen3-Coder-Next? Still and old model, but newer and more capable than Qwen3-coder.
Yep, I totally screwed up. Even coder-next is deprecated at this point. There are just too many models, and it is difficult to keep up with which ones are worth your time and which ones are just old news.
ah, you are exactly right. Coder and even coder-next is useless these days, you should be using 35B-A3B.