Comment by alansaber

As you say- LLMs are fundamentally good because of their generalism. Distillation, ablation, ft all tend to be hacky and in some way hurt the model