Comment by InsideOutSanta
21 hours ago
This line of thinking makes no sense because it assumes that labs that distill from frontier models are doing nothing else. It's the classic "the Chinese can only copy" mentality, and it's going to end poorly for American companies.
I'm pretty sure that all labs are distilling each others' LLMs, maybe apart from Anthropic and OpenAI. It would be stupid not to do it, because it's cheap and effective. But that's not the only thing they're doing. If you think K3 and GLM-5.2 got this good only from distilling frontier models, you're not paying attention to Chinese labs' publications.
i never assumed that, and i do keep up with the publications. i'm also not saying it's a dumb thing to do! what i am saying is that empirically, it appears that distillation of a more advanced model is a required first step for them to train a borderline competitive, cheaper model. in effect, their training is subsidized by the frontier labs.
if this were not the case, then we would be observing chinese models that far surpass frontier models in capabilities, rather than "almost as good, but much cheaper", and we would be having a very different conversation. what happens to these efforts when the subsidy is cut off?
> empirically, it appears that distillation of a more advanced model is a required first step
I see no evidence for that.
> if this were not the case, then we would be observing chinese models that far surpass frontier models
It's pretty clear that the primary reason for the difference is budget and compute availability. Chinese labs have at least an order of magnitude less money than Anthropic and OpenAI.
> what happens to these efforts when the subsidy is cut off?
They will continue making progress as they do now, minus the benefits of distillation.
https://www.anthropic.com/news/detecting-and-preventing-dist...
Moonshot AI Scale: Over 3.4 million exchanges
The operation targeted:
Agentic reasoning and tool use Coding and data analysis Computer-use agent development Computer vision Moonshot (Kimi models) employed hundreds of fraudulent accounts spanning multiple access pathways. Varied account types made the campaign harder to detect as a coordinated operation. We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff. In a later phase, Moonshot used a more targeted approach, attempting to extract and reconstruct Claude’s reasoning traces.
3 replies →