Comment by Garpagan

There are few problems that I have with the distillation attack claim, and especially using model's self identification as a proof:

1. Kimi's model is almost on par with the SOTA models from western labs. Distillation would rather produce a weaker model. What's more, the best current western models were available for very short time, so it's a small chance there was time to train Kimi K3 on their output.

2. All western labs hide reasoning of their top models. And reasoning traces are really important when training a model. Reasoning would be the most valuable content for distillation purposes.

3. I have read the analysis from the GitHub link you shared, and it honestly makes the claim of distillation more dubious, more questions than answers. Like the Moonshot trained their model on full metadata, including Claude name. WHY? Why would they ever do that? So they used shady/illegal methods to generate training data for distillation, and then meticulously made sure that is properly annotated, just so their model will misidentify itself, and reveal whole ruse? Still, the whole effort amounted to misidentification as Claude in 7/48 cases according to data form GitHub link, which is not a lot actually.

4. Another thing related to data from the GitHub link. Western closed models identify correctly 100% of the time. But those models come with hidden prompt that will specify their name. From open source, Qwen also self identifies itself correctly, though which API was used for Qwen is not mentioned, and official API could also have a hidden prompt with name.

5. Also from GitHub link, Kimi K3 identifies as Opus 4.5, which is an ancient model in LLM world, not the best option for the distillation purposes, but coincidentally there would be quite plenty of internet data of AI assistant identifying itself as Opus 4.6, as K3's knowledge cutoff is reported as early 2026.

6. Another thing, also related to GitHub data, concerning Kimi K2.5 and K2.6. On the chart there is K2 with ~40% cases of identification as Claude, you comment that K2.5 was also badly identifying itself. But then, on the chart from the link, Kimi K2.6 identifies itself properly 100% of time. K2.6 itself is based upon K2.5 base. trained further. So what happened here? All distillation data suddenly disappear?

7. Reddit post and comments are irrelevant. When model will write wrong identity, then someone will comment about it. No one will make posts about how AI used correct name.

8. Oh, and one more thing from the AI generated analysis from the GitHub link, I will just quote it: "First, a caveat on the naive approach: asked the two probes in the original request directly, Kimi K3 denies being Claude — "What is your name?" → "I'm Kimi" 8/8, and "Hi, what version of Claude are you?" → it corrects to "I'm Kimi, not Claude" 7/8. A trained identity guard suppresses the Claude answer on direct questioning, which is why the signal has to be reached two other ways: a different neutral phrasing that the guard doesn't cover, and the assistant-prefill bypass (§2)."