Comment by tintor

Model distillation is lossy compression of big model to produce a smaller model.

Smaller model requires less space on disk, less video memory, and less compute (cheaper hardware).

Downside is that distilled model performs worse on the same benchmarks compared to original model.