Comment by ch_sm
5 days ago
Nice, I tried it too with oMLX — agreed, it seems very capable for coding! Was slightly underwhelmed by performance though. I got about ~24 t/s on the ternary version on my M2 Max 64. That’s quite a bit slower than Qwen A3B 35B (4 bit unsloth). How was perf for you?
I had roughly the same performance on M3 Max 64: ~30 tokens/sec. Which is not terrible, but with the latest Lightning MTP optimization I am getting ~100 tokens/sec with Qwen 3.6 35B-A3B.
WHAT. that‘s amazing. thanks so much for sharing, i‘ve been looking for ways to speed a3b up for days. It‘s 11pm here but i‘ll try this right now
I’m using the new oQe quants in case that matters.
2 replies →