Comment by calny
5 months ago
The maintainer's response: https://github.com/chardet/chardet/issues/327#issuecomment-4...
The second part here is problematic, but fascinating: "I then started in an empty repository with no access to the old source tree, and explicitly instructed Claude not to base anything on LGPL/GPL-licensed code." Problem - Claude almost certainly was trained on the LGPL/GPL original code. It knows that is how to solve the problem. It's dubious whether Claude can ignore whatever imprints that original code made on its weights. If it COULD do that, that would be a pretty cool innovation in explainable AI. But AFAIK LLMs can't even reliably trace what data influenced the output for a query, see https://iftenney.github.io/projects/tda/, or even fully unlearn a piece of training data.
Is anyone working on this? I'd be very interested to discuss.
Some background - I'm a developer & IP lawyer - my undergrad thesis was "Copyright in the Digital Age" and discussed copyleft & FOSS. Been litigating in federal court since 2010 and training AI models since 2019, and am working on an AI for litigation platform. These are evolving issues in US courts.
BTW if you're on enterprise or a paid API plan, Anthropic indemnifies you if its outputs violate copyright. But if you're on free/pro/max, the terms state that YOU agree to indemnify THEM for copyright violation claims.[0]
[0] https://www.anthropic.com/legal/consumer-terms - see para. 11 ("YOU AGREE TO INDEMNIFY AND HOLD HARMLESS THE ANTHROPIC PARTIES FROM AND AGAINST ANY AND ALL LIABILITIES, CLAIMS, DAMAGES, EXPENSES (INCLUDING REASONABLE ATTORNEYS’ FEES AND COSTS), AND OTHER LOSSES ARISING OUT OF … YOUR ACCESS TO, USE OF, OR ALLEGED USE OF THE SERVICES ….")
Also the maintainer's ground-up rewrite argument is very flimsy when they used chardet's test-data and freely admit to:
> I've been the primary maintainer and contributor to this project for >12 years
> I have had extensive exposure to the original codebase: I've been maintaining it for over a decade. A traditional clean-room approach involves a strict separation between people with knowledge of the original and people writing the new implementation, and that separation did not exist here.
> I reviewed, tested, and iterated on every piece of the result using Claude.
> I was deeply involved in designing, reviewing, and iterating on every aspect of it.
There was a paper that proposed a content based hashing mask for traning
The idea is you have some window size, maybe 32 tokens. Hash it into a seed for a pseudo random number generator. Generate random numbers in the range 0..1 for each token in the window. Compare this number against a threshold. Don't count the loss for any tokens with a rng value higher than the threshold.
It learns well enough because you get the gist of reading the meaning of something when the occasional word is missing, especially if you are learning the same thing expressed many ways.
It can't learn verbatim however. Anything that it fills in will be semantically similar, but different enough to get cause any direct quoting onto another path after just a few words.
> you get the gist of reading the meaning of something when the occasional word is missing,
I think it's more subtle than that. IIUC the tokens were all present for the purpose of computing the output and the score is based on the output. It's only the weight update where some of the tokens get ignored. So the learning is lossy but the inference driving the learning is not.
Rather than a book that's missing words it's more like a person with a minor learning disability that prevents him from recalling anything perfectly.
However it occurs to me that data augmentation could easily break the scheme if care isn't taken.
Yeah, it's a bit hard to describe what it happening, because the process doesn't really have a human analogue.
People have a difficult enough time dealing with how loss reduction learning is or isn't 'seeing' the data. Selectively removing things from the loss while sill feeding it all the data takes the non-intuitive situation one layer deeper.
That's partially why I described the hash & masking process. I understand it from a formulaic approach but I don't really feel like I have have a good handle of what is happening semantically. It's like thinking in 5D, you can do the calculations but it still feels like your brain is not equipped to deal with what it means.
Thanks! Appreciate the response and will look into this
The difference in indemnification based on which plan you’re on is super important. Thanks for pointing that out - never would have thought to look.
Is this clause even legally valid?
How can the user know if the LLM produces anything that violates copyright?
(Of course they shouldn't have trained it on infringing content in the first place, and perhaps used a different model for enterprise, etc.)
"... If AI-generated code cannot be copyrighted (as the courts suggest) ".
So, Supreme Court has said that. AI-produced code can not be copyrighted. (Am I right?). Then who's to blame if AI produces code large portions of which already exist coded and copyrigted by humans (or corporations).
I assume it goes something like this:
A) If you distribute code produced by AI, YOU cannot claim copyright to it.
B) If you distribute code produced by AI, YOU CAN be held liable for distributing it.
14 replies →
> Is anyone working on this?
There was recently https://news.ycombinator.com/item?id=47131225.
Thanks! I missed that. The attribution by training data source category (arxiv vs wikipedia vs nemotron etc.) is an interesting approach.
Copyright does not cover ideas. Only specific executions of ideas. So unless it's a line-by-line copy (unlikely) there is no recourse for someone to sue for a re-execution/reimplementation of an idea.
Where do derivative works fit into your model of copyright?
It's not "my model." If someone paraphrases a poem, and publishes that paraphrase, the original author will not be able to sue. (Or rather, they can sue, but will almost certainly lose.) There is a body of legal precedent for each category of work you can imagine, and each has come to have its own criteria for what the threshold is for being derivative vs a unique re-expression; but I am confident from how that has played out and from the fact that it is well accepted that code tends to be comprised of only so many patterns, that a codebase that is reverse engineered based on prompting alone will not be considered a derivative work.
It's obviously an opinion. But I'm confident enough in it, as are, say, Lovable and such companies, that I/they are willing to concretely operate on the hunch that that is how it will play out in court if ever the hand was forced.
You've likely paid attention to the litigation here. Regardless of what remains to be litigated, the training in and of itself has already been deemed fair use (and transformative) by Alsup.
Further, you know that ideas are not protected by copyright. The code comparison in this demonstrates a relatively strong case that the expression of the idea is significantly different from that of the original code.
If it were the case that the LLM ingested the code and regurgitated it (as would be the premise of highlighting the training data provenance), that similarity would be much higher. That is not the case.
You're right, I've followed the litigation closely. I've advocated for years that "training is fair use" and I'm generally an anti-IP hawk who DEFENDS copyright/trademark cases. Only recently have I started to concede the issue might have more nuance than "all training is fair use, hard stop." And I still think Judge Alsup got it right.
That said, even if model training is fair use, model output can still be infringing. There would be a strong case, for example, if the end user guides the LLM to create works in a way that copies another work or mimics an author or artist's style. This case clearly isn't that. On the similarity at issue here, I haven't personally compared. I hope you're right.
I think “strong case” is probably reliant on a few points on the output side, and would have to be more than just author/artists style.
Style itself would be very hard to deem infringement, for obvious reasons (idea) - I think it’s much more likely an issue when a character has derivative elements (e.g., iron man, spider man esque features), and where the users prompt had explicit references to those characters (intent)
All that said, even then, on the artistic side I think it would come down to the same analysis that would apply to traditional media - AI is just a vehicle that introduces some novel risks.
Music might be more risky given the litigious nature of the industry.
Code? It’s going to be hard to claim infringement with dramatically different implementations, barring patent coverage.
> The code comparison in this demonstrates a relatively strong case that the expression of the idea is significantly different from that of the original code.
Can I use one AI agent to write detailed tests based on disassembled Windows, and another to write code that passes those same function-level tests? If so, I'm about to relicense Windows 11 - eat my shorts, ReactOS!