Comment by rbbydotdev
2 months ago
Looks like we are seeing small but mighty model breakthroughs, outpacing the pure capital firepower of SOTA providers. I love rooting for the little guy, but is it too soon to call it? To play devils advocate, could it just be the benchmarks are not efficient enough to capture success of real developer workflows?
I think people are going to continue to be surprised by the capability of small models.
Now, if you ask this model to have a conversation with you, it's gonna fail and be incoherent. But boy, does it sure reason through math problems well.
I've just started using qwen3.6:35b a couple days ago running on my framework desktop and rather impressed. It runs really well and reminds me of probably the first Claude model I used. It's the first local model that's actually working for me in a coding agent I've tried. Very exciting!
Try 27b, it's significantly smarter than 35b-a3b (although it is slower, it's not so bad with MTP).
At least according to gertlabs, Qwen3.6 27B outperforms every SoTA (closed) model at Kotlin: https://archive.vn/RYBCL / https://gertlabs.com/rankings?mode=agentic_coding&language=k...
3 replies →
It is, but it's way too slow on a Strix Halo due to its limited bandwidth.
(I'm still sad that they didn't make a 122B-A10B version of it, as it's the kind of model that fits best on a Strix Halo, and for 3.5 it was comparable in performance to the dense 27B version).
2 replies →
Hmm, I just assumed bigger was better. How's it different?
3 replies →
I'm running qwen36.:35b:iq4 IQ4_XS quant. Takes 18 GB of RAM with 131k context window. Seems to be really good. Have it running local stuff via Hermes, using a cloud model via Ollama (Deepseek V4-Pro) for heavy lifting.
If your framework desktop is the 128G Strix Halo, I recommend giving Qwen 3.5 122B-A10B a shot.
This Q5_K_M quant should be near lossless and fit with full 256K context in about 100GB of RAM: https://huggingface.co/AesSedai/Qwen3.5-122B-A10B-GGUF
3.6 scores better on coding across the board.
Edit: specifically Qwen 3.6 27B beats that on coding and agentic workflows.
1 reply →
I'll keep this in mind.
Could you please share which coding agent you are using with it?
Crush: https://github.com/charmbracelet/crush/
The Q8_K_XL MTP model from Unsloth: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF
I settled on opencode after trying goose and aider as well. I'll probably try some more but opencode worked similar to Claude code which is my main agent.
I serve the model with ollama and am thinking about replacing ollama but haven't looked into it.
I have openwebui for chat if I want that too, but don't really use it.
I am using Mistral Vibe.
npx @oh-my-pi/pi-coding-agent
Pi
It feels sometimes like optimizations are only starting.
I’m beginning to suspect the closed SOTA labs were doing all these optimisations, keeping quiet about it, and just charging us out the yinyang for inference.
Also as much landgrab as possible for data centres, infrastructure, that can kepe it running for the next 5-15 years.
Why does an M1 Max continue to remain capable, if not more capable with every passing year with LM Studio? :)