Comment by XajniN
7 days ago
That’s not how it works. The training material in Chinese is more likely to mention DeepSeek. LLMs are not aware what model they are.
7 days ago
That’s not how it works. The training material in Chinese is more likely to mention DeepSeek. LLMs are not aware what model they are.
Of course the model doesn't know what model it is, hence Claude pretending to be deepseek. But whether they got that training content by proxy from the Chinese internet, or distilled directly from the model would not seem to matter for this law.
Do people distil open weights models? Why?
Yes, to make training data for other models. You can distill to get better at something the source model is good at, perhaps it has more training data in a language that you want to support better. Also large open weight models are a great source of training data for smaller models.
That's exactly how it works.
Anthropic says that if a model says they are an Anthropic model, that's it, it's distilled from Anthropic. They even use that as a "proof" DeepSeek is distilling their models.
Or do you want to say the Anthropic PR department is a bunch of liars and scammers?
I think people have been distilling their models... and I also think they're (Anthropic) liars and scammers... and I also think distilling (scraping) a model that was trained on scraped data is just fair turn.
I agree with you, but you also missed the fact that Anthropic is also distilling other models. They would be a bunch of idiots if they didn't do so. Especially chinese open weight models are easy to self host and distill, without having to tell anyone.
[dead]