Comment by irthomasthomas
7 days ago
Of course the model doesn't know what model it is, hence Claude pretending to be deepseek. But whether they got that training content by proxy from the Chinese internet, or distilled directly from the model would not seem to matter for this law.
Do people distil open weights models? Why?
Yes, to make training data for other models. You can distill to get better at something the source model is good at, perhaps it has more training data in a language that you want to support better. Also large open weight models are a great source of training data for smaller models.