Comment by ACCount37

What's in the base model is "bits and pieces of intelligence".

You have to bring those bits and pieces together, put them into the right shapes and fill in the gaps to get a model that actually performs. This is what post-training is all about. It's not at all a trivial thing.

Reasoning, tool use, agentic behavior - all of those are post-training performance gains. Getting a good well trained base model is putting your foot in the door of frontier performance - post-training is how you actually get inside.

See: GPT-4.5 vs o1. One went for "build a bigger better more capable base model", the other went for "take the old base and post-train it for advanced capabilities". The results: a wider base with basic post-training loses to a narrower base with advanced post-training. Or, hell: GPT-3 vs GPT-3.5. One was largely a research lab curio, and the other kicked off the AI revolution as we know it.

The gains compound. Getting a better base model with the same type of post-training helps, see: the jump from Opus to Mythos/Fable. But post-training techniques account for a lot of the performance juice.

And yes, reasoning trace post-training distillation is "genuine distillation". As is logit distillation in pre-training. "Distillation" isn't a single training recipe that you have to follow to a tee - it's a large group of training methods. I've seen plenty of wacky things like inverse distillation bootstrap and post-training self-distillation that use distillation in strange ways at different stages of the training run to get results.