Comment by 3836293648

I was using the 8 bit quant and no reasoning - it’d make mistakes but then fix them at a speed that was impressive - it also was like incredibly tenacious and would honey badger its way around any issues it hit. My second best was Qwen 3 coder next - I did play with 3.5 and 3.6 (both moe and dense variants) but always seemed to go back to GLM 4.7 8 bit mlx variant. I have 128gb mbp so I’ve migrated to Deepseek v4 flash for everything now and haven’t looked back but if a new GLM flash model came out I’d be very excited.

Qwen3.6-35b-a3b at 64k context runs quite well on my 12GB VRAM GPU with MoE partially offloaded to CPU. It does use a good chunk of system RAM too, but I get about 40-50 tok/s.

which quants of 3.5 vs 3.6 did you compare? I guess you're saying that whatever quant you were using, going one lower was worse? ie. 3.5 Q6_K at 22.5GB versus 3.5 Q6_K at 22.9GB?

> 3.6 pushed VRAM usage just out of 24GB and then you're not using a consumer GPU any more

BTW, you can buy an AMD RX 9700 with 32GB VRAM for $1200. Get two of them, and you have a quite powerful local setup. I can run Qwen 3.6 35B at around 80 tok/s and 50% GPU load (300W) and still have plenty of VRAM and power budget left over to run a smaller model for summarization, in parallel.

Highly recommend if you want to play with something that doesn't involve NVidia and/or unobtanium-class hardware.

There were bugs at the beginning (imho worst ones where it kind of works but sucks), you should re-try with latest llama.cpp/quants/whatever you're using.

Stuff like repeated nonsense, endless ???????? output, bogus code, loops after a few hundred tokens, working fine for the first few hundred tokens, then getting stuck in a loop, gibberish output (with flash attention) on after second or third prompt, flash attention failing with kv-cache quantization on long prompts, chat template / jinja / tool-calling problems, inconsistent tool calls in agentic coding, mixed-language nonsense and repeated fragments (corrupted llama-server state / grammar-trigger loop), partial cpu offload/fit problems (it would exit reasoning, start coding, interrupt functions after a few lines, then rewrite snippets repeatedly) etc were all unintended and were fixed.