For me, qwen 3.8 27b on my m5 max is so slow and dumb that I can't think of a single use for it. Haiku, Luna, or data center scale open models are ten times faster and ten times smarter (and inference so cheap that it's essentially free). It was a nifty toy to brag to friends about but why would I waste my time goofing with something so slow and dumb?
Are you insinuating that your agents are more effective at 20 tok/s than 200 tok/s?
I don't even understand your analogy. I'm not watching the youtube video or reading the book. The model is. And, unlike me, a big computer can watch 20 youtube videos faster than a small computer can watch 1, and do a far better job understanding (bigger model at data center scale is both smarter and faster than smaller model at local scale).
Do you find it intelligent enough? What hardware are you using?
For me, qwen 3.8 27b on my m5 max is so slow and dumb that I can't think of a single use for it. Haiku, Luna, or data center scale open models are ten times faster and ten times smarter (and inference so cheap that it's essentially free). It was a nifty toy to brag to friends about but why would I waste my time goofing with something so slow and dumb?
Try DeepSeek 4.1 Flash 1017, feels much better: https://github.com/drawthingsai/local-code-community
For roughly the same reason you might slow down and read a book on how to do something as opposed to speedrunning 20 youtube videos?
Are you insinuating that your agents are more effective at 20 tok/s than 200 tok/s?
I don't even understand your analogy. I'm not watching the youtube video or reading the book. The model is. And, unlike me, a big computer can watch 20 youtube videos faster than a small computer can watch 1, and do a far better job understanding (bigger model at data center scale is both smarter and faster than smaller model at local scale).
On my Mac Mini M4 Pro with 48G I find Qwen 3.8 27B perfectly usable, useful and fast enough.
Are you trying to one shot create the Linux kernel?