Comment by mptest

2 days ago

You can use the outputs of a closed source model (or deepseek -> llama. see llama 70b deepseek distilled) to create a synthetic training data set which lets you fine tune (distill) most of the benefits of the "smarter" model in to a "dumber" model. This is why openAi does not show the actual full chain of thought but a summarized version. To stop exfiltration of their IP which has proven immensely difficult.*

*disclaimer; i am an expert of nothing