Comment by nout
11 hours ago
Multiple teams and efforts are now making Qwen 3.8 27b work much faster on Mac and CUDA and the inference speed is doubling almost every week. There's now ongoing competition about who will make it work faster with exactly the same byte output at https://mlx.fast . TensorFold is the project to follow.
So it's not about the model (eg TensorFold/Qwen3.8-27B-MLX-4bit) it's about the inference engine?