Comment by nickysielicki
7 hours ago
Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a cheaper version of it. There was never any plausible explanation for why this wouldn’t happen. There was never any practical mechanism to prevent someone from saving a conversation and using it to train their own model.
Even if it didn’t happen here, it was still the case that it was going to happen going forward. It was always going to end like this. Invest in the hardware companies, not the model companies.
The desire to accuse China of just copying is like 20 years out of date. It’s been wrong since some people on HN were in diapers.
People are going to be gobsmacked when, in our lifetime, China becomes a world power comparable to the U.S. Probably still poorer per capita, but at Spain/Italy levels, not third world country levels. And they’ll be shocked at the implications of that on the world economy, migration patterns, etc. There will be fields where China is a global leader, and Americans and Europeans will have to learn Chinese and move there, or else be stuck in some satellite office of a Chinese company. We’re all in Europe circa 1895 not realizing the behemoth America will become in WWI.
I strongly agree with the premise that distillation is not an “attack”.
But that said: K3 is not a distilled version of Fable or Sol. Fable has been barely available and Sol was just released! Moreover, K3 is superior to both models in some domains, according to user scoring on the Arena.
API distillation can’t give you these results anyway. All it is useful for is bootstrapping RL in new domains to get past the “cold start” problem faster. By far, what matters more is the quality and variety of RL environments the model learns from.
Almost all markets depend on some form of regulation whether its as simple as "leave everyone alone but no stealing" or "every participant has to source every object through mountains of red tape."
Thus far the US has not really chosen to go the Chinese rare-earth method yet. The problem with distillation attacks is the end result is everyone who is not doing them is going to deal with some kind of regulation whether it's complete loss of access, or the amount of control you'll have to give up to access them will be ridiculous.
Sort of like the "stealing music is fine" but "lets freak out now that it's producing visual art", in the end the entire thing is a social construct. Whether this is treated as theft or "business as usual" is entirely societal.
Eventually the gap will close, unless there's a major breakthrough that hasn't been made yet.
Given these models could not have been trained in the first place if they had to license every line of random fan fiction on the internet, I think distillation also being fair game is a tradeoff everyone should be willing to take (unless they want to decelerate, but that's a different conversation).
Us models didnt pay for licenses too
24 replies →
How is distillation an "attack" but gigascraping the Internet to the point of crashing servers and everyone needs Cloudflare and Anubis now not an "attack"?
I'm not aiming for a what about kickflip here: I'm saying we need to either agree on some rules or stop crying foul. Maybe the coherent legal theory is that neural networks and intellectual property don't interact. That would be weird but it would be consistent, a market could price it, I could do coding stuff and know if I was illegaling.
But this weird gerrymander that no judge will really rule on in an emphatic way is like, bad for the planet, bad for markets, bad business.
There are a lot of reasons to look forward to DeepSeek Huggingface drop kicking the unambiguous frontier weights in like, November, but I think my favorite one will be "who's distilling now bitch?"
I think you've basically got the legal theory. Training a neural network isn't prohibited by copyright law so if you can legally get your hands on something (e.g. by sending a GET request to someone with rights to serve the contents of their web page, or by buying a book) without signing a contract to not train on it, you can train on it.
But the American AI companies only let you query their models if you first sign a contract to not train on the output.
It's hypocrisy and unfair, but I think there's a strong legal argument for it.
Of course China can simply decline to assist in enforcing that contract... But I would expect US courts to do their best to.
15 replies →
Knowlege should not have ownership. Training and distillation should be allowed
Granting people some form of control over knowledge only serves the public interest inasmuch it provides incentive to create more of it. Mass media, effortless duplication, and copyright extensions had already broken this to the point where control of knowledge was suppressing creation of new knowledge more than it facilitated.
The world has changed, we need a mechanism that works for the public interest that applies to the facts as they now are.
I think it's worth stepping back here and pointing out the obvious. Y'all waging war on math. And I'm sorry, but that's the computing equivalent of legislating gravity.
Apologies for repeating myself here, but what you call "distillation" is function approximation.
I feel for the teams at Anthropic and Open AI, but unlike startups from prior eras; Anthropic and OpenAI have decided to be in the business of selling compute. Not creating a product that uses compute, but a product that's math running on compute. This is different from what Google is (or, rather was. As always, RIP Google 1998-2019).
Google's algorithm might be math, but Google search isn't. Google search is a process that's continuously operating in the background. Google crawls pages. Google stores and indexes what it finds. Google then exposes this to retrieval via its algorithm. User uses algorithm.
Now, let's compare that to AI models. When Anthropic serves Mythos / Opus etc, they're taking input or x from their user, doing compute, and then serving the result of the Mythos / Opus function, i.e.,
Where f is a continuous function, https://www.turing.ac.uk/sites/default/files/2025-11/languag...
According to Stone-Weierstrass, given enough values of y for f(x), anyone can approximate this function.
The fidelity and sophistication of this approximation definitely requires a lot of cleverness and effort, and it is arguably an imposition on Anthropic and OpenAI. But on a long-enough timeline, they don't even have to poll Anthropic or OpenAI. As the internet is flooded by PRs, content, emails written by Mythos / Claude, and just people otherwise sharing the results of Claude prompts, then there's an ever increasing set of data to approximate the f(x) that's f_Claude.
Eventually, in the future, anyone will be able to create a good enough approximation of the f_Mythos. Which is Anthropic's product.
Anthropic and OpenAI can now wage war on mathematics and the open-ended compute. Or, they can adapt and build a better product.
Choosing Option B was the Silicon Valley option / choice. I think the OG large-scale Valley lobbying effort, the Semiconductor Industry Association, was unique in that it prioritized and chose to do real research.
https://en.wikipedia.org/wiki/Semiconductor_Industry_Associa...
https://en.wikipedia.org/wiki/Semiconductor_Research_Corpora...
This helped the industry to survive and outcompete the pressure they were facing (at the time).
This is nothing like music piracy.
American labs have ripped everything out of the internet. And now they cry someone else is “stealing” from them. Cry me a river.
Calling distillation an 'attack' is exactly what I've been describing as "AI Exceptionalism":
https://www.magiclasso.co/insights/ai-exceptionalism/
> There was never any plausible explanation for why this wouldn’t happen.
What a nice post hoc revision of history. Distillation is still an active area of research, that you can distill models as easily as you can it genuinely interesting and absolutely not something that was taken for granted even 12 months ago.
Even 6 months ago this idea that 'using model outputs as training examples' was listed as the reason that all models would fail in the near future due to some spooky circular training catastrophe.
Don't pretend like this was so obvious.
I think you’re being overly combative. It’s intuitively quite obvious that it’s incredibly easy to implement and the circular training catastrophe was only ever a conjecture. It’s kind of like releasing a crypto primitive without knowing a proof. Like… maybe it works, but you can’t assume that just because you don’t know how to break it. You have to remember that 100s of billions of enterprise valuation rely on frontier models being moats. The burden of proof is on those raising valuations assuming they will capture the full market.
I agree that hindsight is doing work here, but DeepSeek R1 from Jan 2025 seemed to heavily leverage distillation, and 18 months is an eternity in this climate.
Look how hard Anthropic is to even be able scroll back on your conversation, or look at the thinking tokens or subagents. They want to keep everyone coming back to the watering hole but never to learn how to dig a well.
Did you enable the flicker-free TUI mode?
Why is it hard to scroll?
It really is not, not sure what OP is on about.
I suspect that distillation attacks may be slightly exaggerated. Most of the training data used during fine-tuning is now synthetic data. You can't just repeat the same stuff twice, therefore another LLM is writing a text book that is explaining a topic in detail, ideally without any gaps in the material.
Well, there is precedence: Google can scrape the web, but you can't scrape Google. Laws around compiled databases exist for a reason: you can't just copy the phone book if effort has gone into compiling it, it is itself copyrightable
That varies by jurisdiction. In the United States, copying the phone book (or otherwise copying facts from someone else's collection) has been legal since 1991:
https://en.wikipedia.org/wiki/Feist_Publications,_Inc._v._Ru....
This is the opposite of legal reality, at least as far as the US is concerned.
And funnily enough, most laws about compiled databases might not apply, for one.
And then there's new updates related to AI that fully take out LLMs from protection.
> Distillation “attacks” are not attacks.
Say it louder for the people in the back. All these complaints about "distillation" from frontier labs are bordering on felony contempt of business model at this point. It's great for us. Maybe it's bad for them but nobody other than shareholders really cares.
The optimal outcome for humanity is for oligarchs to spend trillions training a godlike AI, only for the precious weights to just leak. No "distillation" required.
The fact that API based distillation is even a conversation right now makes me feel like the U.S. has their heads so far in the sand that it’s not really excusable.
These Chinese labs are producing novel models, publishing their techniques and sharing their open weights and the first topic of conversation is how they stole from U.S. AI labs.
Setting aside the fact that it doesn’t make any feasible sense to do API distillation, these models are outperforming frontier models on a number of benchmarks, and often times run more efficiently by several orders of magnitude.
We have to stop crying distillation, it’s getting embarrassing and at this point feels even a bit delusional.
There's little doubt that Kimi K3 was distilled off Claude.
Anthropic stated in February that Moonshot AI (the creator of Kimi) distilled ~3.4 million exchanges from Claude models, as explained in their press release https://www.anthropic.com/news/detecting-and-preventing-dist...
While it sounds like a lot, do you suppose 3.4 million sessions come even close to being sufficient to train a frontier model?
Assuming each session was 10,000 words each, that's 34 billion words; lets call it 50 billion tokens (0.05 trillion) unfairly pilfered from Claude. That left Moonshot needing to scrounge for the other 14.950 trillion training tokens required for a baseline frontier model.
2 replies →
> We have to stop crying distillation, it’s getting embarrassing and at this point feels even a bit delusional.
It's a PR campaign - when they say its an "attack" they don't mean on Anthropic - but on America itself. What kind of American can let such a brazen attack go unanswered? At the very least, they ought to demand the dangerous, pinko, stolen models be banned in all 50 states, and pay whatever price demanded by the patriotic, freedom-loving, all-American AI labs that can never be accused of stealing.
or the application layer - which will capture majority of the value.
yeah hardware companies make for nice stories or green numbers on Wall Street - but value will be captured by application layer.
look at history.
That’s true up until the point where you can ask the hardware you made to make its own application layer.
assume you are a "second class lab" and you are in fact making progress by distilling the results of the frontier labs' efforts.
what is the end game for this strategy?
if the frontier labs shut down, or stop releasing to the public, and there's noting left to distill, how will you progress?
This line of thinking makes no sense because it assumes that labs that distill from frontier models are doing nothing else. It's the classic "the Chinese can only copy" mentality, and it's going to end poorly for American companies.
I'm pretty sure that all labs are distilling each others' LLMs, maybe apart from Anthropic and OpenAI. It would be stupid not to do it, because it's cheap and effective. But that's not the only thing they're doing. If you think K3 and GLM-5.2 got this good only from distilling frontier models, you're not paying attention to Chinese labs' publications.
i never assumed that, and i do keep up with the publications. i'm also not saying it's a dumb thing to do! what i am saying is that empirically, it appears that distillation of a more advanced model is a required first step for them to train a borderline competitive, cheaper model. in effect, their training is subsidized by the frontier labs.
if this were not the case, then we would be observing chinese models that far surpass frontier models in capabilities, rather than "almost as good, but much cheaper", and we would be having a very different conversation. what happens to these efforts when the subsidy is cut off?
5 replies →
Distillation from a teacher model solves the self-start problem, that is, building a model to the point where it reason coherently. Without distillation, solving self-start is incredibly difficult since it requires millions of high quality training samples. Creating that kind of dataset takes an enormous amount of effort.
Once a model becomes competent enough to perform complex reasoning, a teacher model is no longer necessary. The model can now reason about its own behavior and build a better version of itself through recursive self-improvement (RSI).
Kimi K3 is capable of RSI.
In public with budgets that don't risk destroying the American economy presumably. Yes it may be slower.
> with budgets
and what will fund these budgets exactly? inference is cheap, distillation is cheap, training is what's expensive.
9 replies →
There doesn't need to be progress at this point. Some models even from 1 or more years ago are useful for some purposes
Thanks for the models guys, sorry for your losses. Once this reality becomes mainstream and undeniable, surely the bubble pops and then what then. Future model development stops? Becomes private? Becomes a public effort?
The existing models are still going to exist. As hardware improves, there will be a day where it might cost a tenth of a penny to churn through 100M tokens a second of Opus 4.8. Established compute providers will invest in improving the models incrementally when margins drive them to look there.
>Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models
So why didnt we have these LLMs in 2005?
Answer the question "how much does 5 cents of LLM computation in July 2026 cost in July 2005" and you'll have the answer to your question.
Don't forget to account for all the costs. It's not just that CPUs are X times slower. Memory is X times smaller, too, and networks are X time slower. And all this hardware is many times more expensive.
If I'm getting my mental estimation right, training a 2026-frontier-class LLM in 2005 would be somewhere on the order of all the computation power in the world at the time. It's not that many more factors of magnitude before you end up at "all the computation power in the world up to that point".
Is this some form of rage bait? 2005 we hadn't the GPUs, we have today. There are other factors, but I think this is the big one. The mathematics of building an LLM are really old, we just hadn't the hardware to do the needed calculations.
Right. Therefor it's not simply a derivative of information. The hardware is required to build the model. Software as well. The model uses information, it is not "distilled" from it.
"Distillation" literally means to separate and take some components out of something. You can distill how a model works from a model. You cant distill a model from information because the information does not contain the model.
People are happy to conflate distilling with building because they dont like how the information was used. You distill how the model works from the model, and you build a model with information. Both could be morally good or bad but its not the same thing.
6 replies →
Because the transformer architecture that enabled modern LLMs wasn't invented until 2017[1]?
1: That's the "T" in GPT fyi, even though Google is the author of the research paper that changed everything
Right. So we had enough information to train LLMs but not the technology to build it.
So the initial models arent just distilled from information. We’ve always had the information.
1 reply →
Moore's Law or something. Were you alive in 2005? The Nintendo DS getting the Opera browser was a big deal. THAT 2005 with today's LLMs? Hilarious.
We didn't have the compute required (GPUs powerful enough to parallelize forward and backward pass). This compute is what allows us to train from human knowledge or distillation.
because you had neither the chips or the information in 2005. You have probably on the order of 5000x to 10000x more GPU compute today than you had in 2005 and three to four magnitudes more openly available data.
The first "L" in LLM does the work. In 2005 you had no Github, Stackoverflow, Youtube, common crawl and no archive of digital ebooks.
> Distillation “attacks” are not attacks.
If "distillation attacks" happen, we have to conclude there is some value add in what model labs do. Regardless of how we feel about using existing human knowledge in the way they currently do, it's simply impractical to infer that everything that happens downstream of LLMs can not be an attack on some IP because of it.
So both things can be true: a) People infringe on Anthropics IP and b) what Anthropic did to build their models is legally questionable (or might be ruled illegal, even though I doubt it).
>People infringe on Anthropics IP
No.
Authors do not infringe on IP when they read another's book, nor should the lumber company be able to dictate how I use planks and if I can resell them if i'm done with them.
You're framing it as if the added value of the author or lumber company, awards them consideration when somebody uses the products to create more value.
IP law was always a big mess, and these questions cross far into ideology instead of law; but I do not understand people who think we need an ideology where more IP-law is good for society.
It's more simple: They infringe on the IP by way of violating the ToS. If you violate ToS and the company suffers financial harm, they usually can (usually) sue you in civil court for damages.
5 replies →
>Authors do not infringe on IP when they read another's book
Are the distillers reading books or are they building models?
If anthropic is providing no value they can just build from scratch. But obviously distilling is easier. Hes saying thats the value they add.
There are some quite interesting legal implications here. If Anthropic has IP over output produced by agents, do they somehow have legal rights to code and documents produced by such agents?
This would demolish agent usage by corporations.
2 replies →
> People infringe on Anthropics IP
Unless someone literally stole the weights somehow (which is not out of the question, I doubt either oAI/Anthropic have the capabilities to prevent a state-level actor getting those weights), distillation from generations is not infringement on anyone's IP nor is it stealing nor is it an attack. It can't be. As long as you pay for tokens you get to do whatever you want with them. Someone saying you can't doesn't mean it's an attack or their IP or whatever. They either sell the tokens or not. They can decide to not sell them to anyone, but again that's not stealing.
And their ToS are a joke. Imagine how people would react if MS had ToS saying that you can't use MS software to develop solutions that compete with MS. They'd be laughed out of the room. Somehow it's ok for token sellers to decide what you do with the tokens? Why? If you pay for something you get to do whatever you want with that output. Train, distill, whatever.
Its definitely an attack. Thats established from anthropics perspective. No one has a right to use Anthropic’s services in ways that directly violate the ToS and user agreements.
2 replies →
> People infringe on Anthropics IP
Anthropic’s model outputs contain no IP. This is actually a simple legal proposition (rare in this field!) that derives from the fact that only specific classes of IP exist: copyrights, patents, trade secrets, and trademarks. Examining each, it is clear that API outputs do not qualify. Anthropic disclaims copyright in outputs; the outputs are not patented; the outputs are not secret (a prerequisite to having trade secrets); and trademarks are irrelevant in concept.
The output of Anthropic's models is not Anthropic's IP, as that would destroy their market, if Anthropic owned all the software it generated, and all the content. So distillation, which is just using those outputs is always going to exist.
I'm pretty sure that LLM output is not intellectual property. Nobody owns it, and it can't even be copyrighted. So using output from Anthropic's LLMs in ways Anthropic does not condone is not IP infringement.
Whether or not its legal to distil models, it is obviously morally permissible to do so.
Anthropic, OpenAI, etc do not deserve legal protection.
anthropic model output is not their IP
that would be existential doom for them because then they have a case to claim ownership of their users' codebases
no corporation would sign off on that
Considering they were the original infringers, I don't know how anyone can expect tears to be shed here. The best we can hope for is for all these cancerous - and they really are the definition of a cancer - money burning entities to all fall apart to distillation attacks like these.
The value is simply that it is easier. The same way it is easier to ask someone who has experience for advice than reading hundreds of textbooks.
Regardless of whether it’s intellectual property or it isn’t intellectual property, it doesn’t actually matter. If AI doesn’t stop seeing diminishing returns in scaling up, and it hasn’t yet in the 10 years since the attention/transformers paper, the advent of AI will be the most important development in the history of humanity. Controlling that machine, or at least having one of your own, is an existential problem for nation states. It’s like a matter of national defense.
Do you really think intellectual property laws will prevent this in practice? It’s like as if we said, “hey, USSR, you can’t make a nuke, too! We patented that already.”
Asking China to not distill our models down is equally as ridiculous.
I do like that you mention diminishing returns, because we are hitting them in building out all the external requirements for competing at the frontier. Even if model performance scales linearly with energy input, the top labs are now competing with other uses for that energy.
How far are we willing to go as a nation (and as a species) to prove out the scaling laws? Are we willing to sacrifice our industrial base? Would we rather train models or smelt aluminum?
It's unlikely the USA would be granted an exclusive patent for the atomic bomb given the well-established existence of prior-art in the form of nuclear fission on the sun.
(I actually appreciated your analogy, despite my lark)
In fact, China stealing fire from the gods is essential to the future balance of power, so long as they keep making the results freely available.
Anthropic’s IP is basically null and void for how they created it. And they might not want to try and challenge this in court, considering how they had to settle for using text books they had no right to use