Comment by iamnotagenius

Fine-tuning is excellent way to reliably bake-in domain specific data into a model; there is a plenty of coding finetunes on Huggingface face, that outperforms foundation models on say coding, without significant loss in other domains.