Comment by FooBarWidget

Distillation helps, but is only a minor part of overall training effectiveness. It's not like everything suddenly collapses if distillation is made impossible.