Comment by bandrami

I think the idea is you sink the pretraining costs once and then you can distill multiple specialized models from that