Comment by teravor

distilling K3 into DS4 flash will likely only be a good idea for specialists. the difference in model capacity is otherwise too large.

we used to distill GLM 5.2 into Qwen 27B specialists with great success.

developing the workflows is tricky though. we had the advantage of a straightforward mapping function in mind: English -> SQL which was relatively easy to pipeline training for.