Comment by raffraffraff
5 hours ago
Here are the numbers that got me buying a used M1 Max for LLM work.
M1 Max 400GB/s
M4 Pro 273GB/s
M5 Pro 307GB/s
Only Max are worthwhile for LLMs. An Ultra is just two Mac chips on a die, but I think they're just used in Mac Studio.
I'll check it that inference engine! Thanks!
Edit: it doesn't give numbers for unsloth/Qwen-3.6-35B-A3B 4bit, but I'm using mlx-serve right now and getting petty good numbers (and it took a while to tweak everything properly!)
it has just a few models it can download, it's really qwen and apple silicon pilled
it has unsloth on the list of 3.6 35b variants only on the 2-bit unsloth UD-IQ2_XXS GGUF variant
I'd say let it down 4-bit Qwen 3.6 35b again and see what how it performs in both benchmarks and your preferred conversational and agentic harness
it's Kflash verifier is competing against MTP and Dflash2, thats where the additional novelty is