Comment by jeffyaw
10 hours ago
disagree, it’s absolutely comparable. but an m5 max is where it shines. and there are 50 flags to finetune with llama server before you get it right. i do prefer llama server over mlx currently on mac as well.
10 hours ago
disagree, it’s absolutely comparable. but an m5 max is where it shines. and there are 50 flags to finetune with llama server before you get it right. i do prefer llama server over mlx currently on mac as well.
What, I run it on 2 H100`s with vLLM and this is where it really shines. It's about half as good as Sonnet 5.5. On a small m5 max I can only assume trouble.
It’s so poor for most tasks that you’re faster doing it manually.
DeepSeek and GLM are useful, though. Even the flash variants.
I have all three and you're not going to convince me of that.
Side-by-side Qwen 3.8 27B is ~6x as fast as DS4 and GLM 5.3 and faster than that if you get the speculative engine set up properly. That means you can iterate very rapidly on the same problem and for many problems this is the way to solve them. DS4 and GLM 5.3 shine when Qwen would take too many turns to solve something, they simply have more world knowledge and seem to home in on possible solutions faster in #tg but slower in wall clock time. If a particular problem is within striking range of all three then Qwen will usually be the fastest solver by far.
It also depends on how good your prompt is, whether you are setting up points for guidance or whether you expect everything to be solved in a single shot.
> Side-by-side Qwen 3.8 27B is ~6x as fast as DS4 and GLM 5.3
Wow, that sounds like a lot! Do you use/mean the regular models or flash variants here?
1 reply →