← Back to context

Comment by Casteil

3 days ago

I'm hoping too that they'll put out some MoE variants.

Qwen3.5:122b:a10b can run about twice as fast as this 27b dense model.

Edit: Like its predecessors, 3.8 seems really inclined to overthinking, and on a 27b dense model that's kind of painful. I think I'm going to stick with gemma4:26b-a3b as my go-to because it runs about 4x as fast and tends to only need a fraction of the tokens in its 'thinking' stage to get the same or similar answer.

12 comments

Casteil

Reply
isomorphic  2 days ago

"inclined to overthinking," holy cats you're not kidding! On a Mac mini M4 Pro 64GB I prompted it with "svg owl" and it thought for 17m12s, outputting 36.3KiB of thinking chatter. It did end up producing a 20.2KiB HTML+JS+SVG file with a very nice owl, including cursor-tracking animation, but it ran for more than a half hour!

The MoE models are stupendously faster.

  • kzrdude  2 days ago

    I think that's called test-time scaling i.e using more tokens at infer time to squeeze out higher model performance. That's must be part of the explanation for good benchmark results.

    • try-working  2 days ago

      that seems to be how most Chinese models achieve increased benchmark scores. GLM and Kimi models are "thinkslop" models that reason over their own thinking, which increases cost and decreases speed significantly. That's why GPT is in a different tier altogether - faster, smarter, and sometimes cheaper.

    • Casteil  2 days ago

      Yep.. it's pretty obnoxious for real-world use with the default 'xhigh' thinking. Ridiculous amount of "Wait, actually.." which might help for complex coding tasks but makes it unbearable for general purpose use.

  • nubg  2 days ago

    what harness did you use?

    • isomorphic  2 days ago

      No agentic harness; I just prompted literally "svg owl" into LM Studio 0.4.21+2. macOS Tahoe 26.6.1, Mac mini M4 Pro (14 core, 20 graphics cores) 64GB RAM. (I put the machine config so the 17m12s of thinking has some context.) It got about 12.75 tok/s and used 21,769 tokens.

      For comparison's sake, Qwen 3.6 35B A3B MLX, same prompt, same machine, thought for 1.59 seconds, used 2,398 tokens, and hit 80.83 tok/s. It actually output just a straight SVG file of an owl (not a bad owl either!), whereas Qwen 3.8 27B spent a zillion times longer gold-plating everything and output HTML+JS+SVG with interactive animation, sound, etc., etc.