Comment by stratos123

Distillation lets you train a comparable model for a fraction of the cost (it's effectively a way to get a lot of very-high-quality training data). If you're already at the frontier, pushing it requires the ordinary, expensive kind of training.