Comment by anon373839
19 hours ago
I strongly agree with the premise that distillation is not an “attack”.
But that said: K3 is not a distilled version of Fable or Sol. Fable has been barely available and Sol was just released! Moreover, K3 is superior to both models in some domains, according to user scoring on the Arena.
API distillation can’t give you these results anyway. All it is useful for is bootstrapping RL in new domains to get past the “cold start” problem faster. By far, what matters more is the quality and variety of RL environments the model learns from.
API distillation doesn't have to explain all of K3's capabilities for it to have happened. Kimi K3 reproducibly identifies itself as Claude: https://x.com/denisewu/status/2077984660211269870
This behavior is exactly what you'd expect from a model distilled from Claude.
There's a detailed analysis of K3's ambiguous identity here: https://github.com/rgreenblatt/which_claude_is_k3/blob/main/...
This analysis observed K3 identifies itself as Claude approximately 15% of the time.
K3 reproduces Claude's correct current model id, which the real Claude models themselves do not emit. This suggests K3 was trained on Claude data labeled with deployment metadata (API logs, tagged synthetic data), rather than Claude's chat outputs.
And there's an entire Reddit thread discussing Kimi's similarities with Claude https://www.reddit.com/r/LocalLLaMA/comments/1m2w5ge/did_kim...
This analysis shows K3 and Opus/Fable have unexpected correlated outputs https://typebulb.com/u/lab/you-re-relatively-right/full
Kimi calling itself claude means nothing. During pre-training, when the model learns to "simulate" the internet text, it will naturally be fed with a bunch of data about Claude and ChatGPT. With the amount of LLM outputs on the internet today, it is not surprising at all that a model would naturally call itself Claude or ChatGPT. You can mitigate that in post-training (or actually in pre-training as well) by training on many examples of what the model should call itself. That being said, getting probably hundreds pf thousands of ChatGPT and Claude examples totally "pirged" out of the weights is going to be difficult and really more hassle than its worth.
Sure, but then Qwen should leak that too, and it doesn't. K3 calls itself Claude 7 out of 48 times, Qwen does it 0 out of 48, and the only other model to identify itself as Claude is DeepSeek. and DeepSeek is alleged to also distill from Claude data anyway. So this isn't something every model absorbed from the same web text.
And you skipped over the strongest datapoint that K3 is distilled: K3 reproduces Claude's public model identifier under prefill (i.e. "claude-opus-4-5-20251101"). This data does not appear in Claude chat logs, only in API logs. K3 only does this for Claude models and not for any other lab. The real Claude models don't produce their own current public identifier, they only know their previous identifier (i.e. Sonnet 4.5 calls itself "Claude 3.5 Sonnet").
This is highly suggestive of the type of data that K3 was trained on. K3 was very likely trained on Claude metadata traces (API logs, tagged synthetic data). Not web chat logs, those wouldn't include this identifier. And this data wasn't filtered correctly, which is why K3 incorrectly identifies itself as Claude 15% of the time.
You can also look at the last link and it's pretty damning: Kimi K3's output has an uncanny similarity to Fable/Opus output. https://typebulb.com/u/lab/you-re-relatively-right/full
6 replies →
> Kimi K3 reproducibly identifies itself as Claude
It could also be have been trained from collected response datasets. Claude got caught several time responding it was ChatGPT or even Deepseek and I don't think Anthropic has been distealling DeepSeek.
> This behavior is exactly what you'd expect from a model distilled from Claude.
The opposite actually. If they wanted to distill Claude without getting caught they could just use a regex to change Claude to Kimi in their distillation pipeline!
Apt typo.
Though I am of the opinion that distilling is no different than how extant frontier LLMs have also been trained on other people's data, I could actually see the word distealling becoming useful in discussion.
1 reply →
> they could just use a regex to change Claude to Kimi in their distillation pipeline!
Jean-Kimi Van Damme would like to have a word with you.
and claude will call itself chatgpt etc.
nothing new, all ai labs are immoral and not bound by any reasonable oversight or ethical constraints. All outlaws in their own rights on that front. Absolutely none of them have true rights on the matter of being distilled from given historic and continued behaviour. I'm not sure why this is a talking point at all? We know AI companies steal, the least interesting behaviour among this is them stealing from one another.
For me, a far more interesting and important point of conversation on this matter is anthropic buying rare or evwn unique books, processing them for training data, and then destroying the books for others cannot use it as well.
Permanemt destruction of priceless primary source materials is so many leagues beyond copying a copy that I cannot fathom it even registering as a discussion point.
All large corporations are immoral FWIW
> For me, a far more interesting and important point of conversation on this matter is anthropic buying rare or evwn unique books, processing them for training data, and then destroying the books for others cannot use it as well.
That's an incredible allegation, and appalling if true. But is it true?
1 reply →
It does not reproducibly identify itself as Claude, there's evidence to the contrary in the very thread you linked: https://x.com/bobbyNewcomb5/status/2078151562828947954
As mentioned in my comment, Kimi K3 identifies itself as Claude ~15% of the time.
Here's another report of K3 identifying itself as Claude https://x.com/Sauers_/status/2077842686459981901
And an analysis showing the self-identity distribution for K3 and other models https://x.com/RyanGreenblatt/status/2078663148509544589
45 replies →
But then again, the identity could also have slipped into the model from other sources during pretraining. The internet is full of "I am Claude": https://grep.app/search?q=i+am+claude and variants https://grep.app/search?q=i%27m+claude
Either way, there's probably no significant portion of Mythos/Fable or Sol in there as OP has stated.
Surprising they didn't clean that from the data before training. It's easy to identify, a simple search->replace gets most of it, and a cheap LLM can identify the edge cases (e.g. avoiding "Claude Shannon" -> "Kimi Shannon" or something).
fwiw, Gemini 3.5 has identified itself to me as an OpenAI product on multiple occasions.
Early Grok would also identify as ChatGPT. This has happened with new model releases for years now.
Claude Sonnet 4.8 reproducibly identifies itself as DeepSeek when asked in Chinese:
https://x.com/stevibe/status/2026227392076018101
I mean, people can point fingers however they want, and the fact is nobody actually "owns" the data they feed to their LLMs...
The entire debate about distilled vs not distilled is just academic. As an end user, I don’t give a crap how the model was trained as long as it does the job affordably enough
I see all of AI as theft anyway so it makes no ontological difference if the theft was from a human or from another AI
If your business leaks to your customers… if you can ask your product for its inner secret sauce and it readily gives away the goose. That is not a moat.