Comment by didibus

Well, most LLM are fine tuned over higher quality data, this is kind of how they've kept improving them amongst other things.

The first pass is to learn the fundamentals of language, and then it is refined on curated datasets, so you could refine them on high quality curated C code.