Comment by weitendorf

You 100% can finetune or adapt/build on top of models, and specialize them or extend their capabilities. That’s literally what post training is.

The problem is that “finetuning” was a 2023 AI FOTM associated with products/demos that were almost exclusively using it for LLM character role-play/output style purposes (ie not in actual systems where they served a more functional role).

This made people think you could train models without replay/real evals by yoloing it with SFT (this is partially an artifact of that era being much heavier on autoregressive training and not so much evals). You really can finetune and get results but you have to treat it like a small ML training run, with real evals, and more intentionality than just “more examples”.

You can find pretrained and -instruct models on huggingface that clearly demonstrate what specialization/staged training runs do.

I’d be very wary of conflating finetuning with specialization/extending a model’s capabilities in general.