Comment by margalabargala

Their point is that "training" and "distillation" are essentially the same. The difference between the words is whether the source material is output from another model, vs being some original text.