Comment by petesergeant

This doesn't seem to be true. There's a clear legal path via the first-sale doctrine to train models on copyrighted works. It's been years now, and publishers still don't seem to be offering anything for training (e.g. bulk licenses solely for training use), but adversarial interoperability via cutting up books and scanning them remains perfectly legal.

There's also the ability to distill other models, which is also not illegal (though I'm sure they like to come after whomever for TOS violations, but thats a civil matter).

And, of course, the obligatory copying-isn't-theft observation. A recent supreme court judgment put it well.

> Since the statutorily defined property rights of a copyright holder have a character distinct from the possessory interest of the owner of simple “goods, wares, [or] merchandise,” interference with copyright does not easily equate with theft, conversion, or fraud. The infringer of a copyright does not assume physical control over the copyright, nor wholly deprive its owner of its use. Infringement implicates a more complex set of property interests than does run-of-the-mill theft, conversion, or fraud.

Folks are pretty smart here, I think we can handle these nuances, even if we don't agree about whether they are good.

Edit: reading through the full text of their post, it looks like they are using common crawl, which is likely just as much of a copyright infringement as Anna's Archive -- it's not like published works have a unique claim to copyright. I think this strengthens your point, though: I was expecting to see scans as training data, but it doesn't appear to be the case.

Disclaimer: I am part of the team that trained Kolibri, opinions are mine.

You're right that especially big models benefit from training on copyrighted material in terms of world knowledge (especially from books). However, in the small model space imho agentic capabilities where the model looks up knowledge on the fly are much more important. That's what we focused on quite a bit during training. Personally, I also don't think stealing stuff is okay.

Why should we concede this "stealing" framing? If I want to put texts into my computer program, why should that count as copyright violation? I think that idea is as ridiculous as saying that reading a document is a copyright violation.

Regulate large cloud services and proprietary software - yes! But not on the basis of "Intellectual Property".

The "legal" issues here are very very complex and we should not passively wait for or accept corrupt court rulings, international trade agreements, proposed laws, or worst of all propaganda that pushes a parochial and craven view on this.

This model isn't terrible, at least on the benchmarks. It's 78B A3B and performs about like Qwen3.6 35B A3B. You can probably run it comfortably in 96B of RAM with a decent quant that doesn't lose too much.

Unfortunately, Qwen3.6 35B A3B isn't really a useful coding model. You'd probably want Qwen3.8 27B at a minimum, which requires at least 32GB of VRAM (not system RAM) to run semi-comfortably.

So this isn't going to be a competitive model for hobbyists, and you'd have to be a bit desperate to use it for coding. But if you work in a regulated industry and don't mind paying for a bit of extra hardware, it isn't catastrophically bad, either. Probably would work fine for information extraction or as a "classifier" like Jev. (Almost any GGUF model can be turned into a classifier using llama-server. See pi.dev codemode for sample code.)

So they're not a real contender yet, but they look like they're probably at least minimally credible.

hasn't IP law passed the statute of limitations? As in most models are probably trained on output of other models, as creating enough data otherwise is not feasible. Additionally, they are trained on github repos made since the AI boom, which were generated by models with IP issues (who knows what and how).

Thus training on 'clean' data is like trying to unscramble an egg.

There is no word for copyright in Mandarin :)

Not really. The upside of competitive newer models is all in the proprietary data they are trained on. This is why data labeling, RL environments et al have been such a big industry, OpenAI and Antrhopic are paying literally billions to get the data they need. Do people think the ability to do research level math or advanced cybersec comes from just training on more public data?

A real sovereign effort could invest heavily in this, whatever people accuse China of “stealing” I’m sure they are also generating tons of their own data and are probably the primary sovereign doing so outside the US labs.

Have you tried the model itself and seen if it's "even slightly competitive" or not, and have specific complaints about it? Otherwise it feels like you're complaining about something that is easy to test but rather than taking the time to actually figuring that out first, you're arguing about some general and theoretical thing which the submission (may) directly disprove.

Is it noble? The entire notion that training data can be "stolen" at all is quite silly. If I "steal" content that someone created to use for training, what am I actually stealing? They didn't lose anything. They still have everything they had before. What was "stolen" was "unrealized profit", or put another way: money that wasn't theirs, that they had no entitlement to. The only actual crime that is committed is "unauthorized copying", not stealing. Support and enforcement of copyright feels wildly authoritarian. It's hard to see it as noble.