Comment by simianwords
1 day ago
To everyone praising Open weight models, could you answer a simple question?
If Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them, such that it makes financial sense for Anthropic to train even bigger models?
Assuming it is preferable for everyone that we get better models in the future. Distillation attacks remove the financial incentive.
This assumes all the open models are just a result of distilling Anthropic models. Which remains to be proven.
And if they are, the point remains that Anthropic has a brittle product advantage that users and investors should be cautious about.
OpenAI says they're not just distillation. "Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that."
If you put any faith in claims made by OpenAI's "Head of Strategic Futures," then it's hard to believe these models are just the result of distillation.
> investors should be cautious about
if they are cautious, what would make them invest in newer bigger models without the expected return? generosity?
Sunk cost fallacy.
So much money has been invested in Anthropic and OpenAI at this point that to declare it a loss and walk away could potentially destroy a lot of VC firms, and a non-trivial chunk of the US Economy.
4 replies →
I don't think the timeline is plausible for Moonshot to have done any distillation from Fable, which means any distillation training data would have been from Opus.
The frontier labs have been tightly controlling model access lately to prevent further distillation, and the Chinese labs are still catching up rapidly - it's likely that whatever moat they used distillation to overcome early on is already dry
[Pending evidence that Chinese models are only based on distillation]
While I am sure some do distillation (even r/LocalLLaMA is full of randos bragging about it), I wish the companies named by Anthropics would respond to these accusations.
I think at least deepseek can easily take Anthropics to court and win a defamation case. The number of requests allegedly done from deepseek IP range is so small it is barely enough to run a few benchmarks.
> f Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them,
I dont care if that company dies. Or if OpenAI dies. I dont mind investors investing into other things either.
You can simplify your question even more:
To everyone praising free and open source software, if software companies don't make money, how are they and YOU going to get paid, PERIOD?
Commoditising complement