Comment by docheinestages
6 hours ago
Is there a pipeline or approach to do this to any model? I'm particularly interested in Qwen 3.6 27B as it's the best for its size at the moment.
6 hours ago
Is there a pipeline or approach to do this to any model? I'm particularly interested in Qwen 3.6 27B as it's the best for its size at the moment.
This approach will only work for MoE models. There is a Qwen 35b-a3b. You just need to do GPU stop after router and read the requested experts to ram. And it is possible to build similar engine for this model (or feel free to adopt my engine) Not sure about generic approach for now, but coding with ai agents is relatively cheap now, you can try it