I believe that Pilgrim here does not understand very well how copyright works:
> Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code
This is simply not true. The reason why the "clean room" concept exists is precisely since actually the law recognizes that independent implementations ARE possibile. The "clean room" thing is a trick to make the litigation simpler, it is NOT required that you are not exposed to the original code. For instance, Linux was implemented even if Linus and other devs where well aware of Unix internals. The law really mandates this: does the new code copy something that was in the original one? The clean room trick makes it simpler to say, it is not possible, if there are similar things it is just by accident. But it is NOT a requirement.
Regardless of the legal interpretations, I think it's very worrying if an automated AI rewrite of GPLed code (or any code for that matter) could somehow be used to circumvent the original license. That kinda takes out the one stick the open source community has to force soulless multinationals to contribute back to the open source projects they use.
I’m genuinely surprised to see this not discussed more by the FOSS community. There are so many ways to blow past the GPL now:
1. File by file rewrite by AI (“change functions and vars a bit”)
2. One LLM writes a diff language (or pseudo code) version of each function that a diff LLM translates back into code and tests for input/output parity
The real danger is that this becomes increasingly undetectable in closed source code and can continue to sync with progress in the GPLed repo.
I don’t think any current license has a plausible defense against this sort of attack.
It's worrying, but it's consistent with how copyright law is currently written. Laws haven't caught up with what technology is currently capable of yet. The discussion should be whether, and if so how, our laws should be tweaked to stop this from getting out of hand, IMO.
If the AI is good enough to truly implement the whole thing to a similar level of reliability without copying it then who cares. At that point you should be able to decompile any program you want and find enough information inside that an AI can go write a similar quality program from the vague information about the call graph. We've transcended copyright in computer code.
If it can't and it costs a bunch of money to clean it up then same as always.
OTOH if what is actually happening is just that it is rewording the existing code so it looks different then it is still going to run afoul of copyright. You can't just rewrite harry potter with different words.
Note that even with Google vs oracle it was important they didn't need the actual code just the headers to get the function calls were enough. Yes it's true that the clean room isn't required but when you have an AI and you can show that it can't do it a second time without looking at the source (not just function declarations) that's pretty strong evidence.
Take AI out…if a person can do it, which they can, the situation hasn’t changed. Further, it was a person who did it, with the assistance of AI. Also, the concept that you “can’t be exposed to the code before writing a compatible alternative” is utterly false in their arguments. In fact, one could take every single interface definition they have defined to communicate and use those interfaces directly to write their own, because in fact this i(programmatic) interface code is not covered by copyright (with an implicit fair use exemption due to the face the software cannot operate without activating said interfaces). The Java lawsuit set that as precedent with JDK. A person could have absolutely rewritten this software using the interfaces and their knowledge, which is perfectly legal if they don’t literally copy and re-word code. Now, if it IS simply re-worded copies of the same code and otherwise the entire project structure is basically the same, it’s a different story. That doesn’t sound like what happened.
Finally, how exactly do people think corporations rewrite portions of code that were contributed before re-licensing under a private license? It is ABSOLUTELY possible to rewrite code and relicense it.
Edit: Further, so these people think you contribute to a project, that project is beholden to your contribution permanently and it can never be excised? That seems like it would blatantly violate their original persons rights to exercise their own control of the code without those contributions, which is exactly the purpose of a rewrite.
It cuts both ways. You can write a GPL version of a proprietary or permissively licensed program. The only difference is the effort of the rewrite is (theoretically) easier.
(I have my doubts the rewrite is a reasonably defect free replacement)
It’s less worry to me given that a year ago this would have been exceptionally harder to do, requiring a lot more time and effort and been more costly. A year from now it will be even easier. All of this means that one aspect of the mission that brought about the need for a license like this is now fundamentally easier whether or not the license is used. There can be less worry about software locked up in closed source overall.
True, but if that is found to be how it works then an automated AI rewrite of closed-source code is just as unbound by the original license. Which is a much bigger win for the open-source community, since any closed-source software can become the inspiration for an open-source project.
Soulless multinationals often want to share costs with other soulless multinationals, just like individuals do. So I think there will always be publicly shared code. The real question is whether this code will be worth much if it can be implemented so quickly by a machine.
That kinda takes out the one stick the open source community has to force soulless multinationals to contribute back to the open source projects they use.
I'll trade that stick for what GenAI can do for me, in a heartbeat.
The question, of course, is how this attitude -- even if perfectly rational at the moment -- will scale into the future. My guess is that pretty much all the original code that will ever need to be written has already been written, and will just need to be refactored, reshaped, and repurposed going forward. A robot's job, in other words. But that could turn out to be a mistaken guess.
Neither does the maintainer that claims a mechanical test of structural similarities can prove anything either waybwith regard to whether legally it is a derivative work (or even a mechnaical copy without the requisite new creative work to be a derivative work.)
And then Pilgrim is again wrong by saying that the use of Claude definitively makes it a derivative work because of the inability to prove it the work in question did not influence the neurons involved.
It is all dueling lay misreadings of copyright law, but it is also an area where the actual specific applicable law, on any level specific enough to cleanly apply, isn’t all that clear.
I think this is a bit too broad. There are actually three possible cases.
When there is similar code, the only defense possible to prove that you have not copied the original is to show that your process is a clean room re-implementation.
If the code is completely different, then clean room or not is indeed irrelevant. The only way the author can claim that you violated their copyright despite no apparent similarity is for them to have proof you followed some kind of mechanical process for generating the new code based on the old one, such as using an LLM with the old code as input prompt (TBD, completely unsettled: what if the old code is part of the training set, but was not part of the input?) - the burden of proof is on them to show that the dissimilarity is only apparent.
In realistic cases, you will have a mix of similar and dissimilar portions, and portions where the similarity is questionable. Each of these will need to be analyzed separately - and it's very likely that all the similar portions will need to be re-written again if you can't prove that they were not copied directly or from memory from the original, even if they represent a very small part of the work overall. Even if you wrote a 10k page book, if you copied one whole page verbatim from another book, you will be liable for that page, and the author may force you to take it out.
> When there is similar code, the only defense possible to prove that you have not copied the original is to show that your process is a clean room re-implementation.
Yes, but you do not have to prove that you haven’t copied the original; you have to prove you didn’t infringe copyright. For that there are other possible defenses, for example:
- fair use
- claiming the copied part doesn’t require creativity
- arguing that the copied code was written by AI (there’s jurisdiction that says AI-generated art can’t be copyrighted (https://www.theverge.com/2023/8/19/23838458/ai-generated-art...). It’s not impossible judges will make similar judgments for AI-generated programs)
The burden of proof is completely uncharted when it comes to LLMs. Burden of proof is assigned by court precedent, not the Copyright Act itself (in US law). Meaning, a court looking at a case like this could (should) see the use of an LLM trained on the copyrighted work as a distinguishing factor that shifts the burden to the defense. As a matter of public policy, it's not great if infringers can use the poor accountability properties of LLMs to hide from the consequences of illegally redistributing copyrighted works.
The expected functionality of chardet (detect the unicode encoding) is kind of fixed - apart from edge cases and new additions to unicode, you'd expect the original and new implementations to largely pass the same tests, and have a lot of similar code such as for "does this start with a BOM".
The fact that the JPlag shows such a low %overlap for an implementation of "the same interface" is convincing evidence for me that it's not just plagiarised.
If you let an LLM merely rephrase the codebase, that's like letting it rephrase the Harry Potter novels. Which, I'm pretty sure, would still be considered a copy under copyright law, not an original work, despite not copying any text verbatim.
But what if it didn’t summarize Harry Potter? What if it analyzed Harry Potter and came back with a specification for how to write a compelling story about wizards? And then someone read that spec and wrote a different story about wizards that bears only the most superficial resemblance to Harry Potter in the sense that they’re both compelling stories about wizards?
This is legitimately a very weird case and I have no idea how a court would decide it.
"Tainted rewrite" isn't a legal concept either. You have to prove (on balance of probabilities - more likely than not) that the defendant made an unauthorized copy, made an unauthorized derivative work, etc. Clean-room rewriting is a defense strategy, because if the programmer never saw the original work, they couldn't possibly have made a derivative. But even without that, you still have to prove they did. It's not an offence to just not be able to prove you didn't break the law.
As other pointed out, the notion of "clean room" rewrites is to make a particularly strong case of non-infringement. It doesn't mean that anything other than a clean room implementation is an infringement.
This is interesting and I'm not sure what to make of it. Devil's advocate: the person operating the AI also was "trained with the code," is that materially different from them writing it by hand vs. assisted by an LLM? Honestly asking, I hadn't considered this angle before.
Given that LLMs were trained on the repository directly, it's not just the case that anything made by the LLM is a derivative work, the LLM ITSELF is a derivative work. After all, they all are substantially based on GPL licensed works by others. The standard courts have always used for "substantially based" by the way, is the ability to extract from the new work anything bigger than an excerpt of the original work.
So convincing evidence, by historical standards, that ChatGPT, Gemini, Copilot AND Claude are all derivative works of the GPL linux kernel can be gotten simply by asking "give me struct sk_buff", then keep asking until you're out of the headers (say, ask how a network driver uses it).
That means if courts are honest (and they never are when it comes to GPL) OpenAI, Google and Anthropic would be forced to release ALL materials needed to duplicate their models "at cost". Given how LLMs work that would include all models, code, AND training data. After all, that is the contract these companies entered into when using the GPL licensed linux kernel.
But of course, to courts copyright applies to you when Microsoft demands it ($30000 per violation PLUS stopping the use of the offending file/torrent/software/... because such measures are apparently justified for downloading a $50 piece of software), it does not apply to big companies when the rules would destroy them.
The last time this was talked about someone pointed out that Microsoft "stole", as they call it, the software to do product keys. They were convicted for doing that, and the judge even increased damages because of Microsoft's behavior in the case.
But there is no way in hell you'll ever get justice from the courts in this. In fact courts have already decided that AI training is fair use on 2 conditions:
1) that the companies acquired the material itself without violating copyright. Of course it has already been proven that this is not the case for any of them (they scraped it without permission, which has been declared illegal again and again in the file sharing trials)
2) that the models refuse to reproduce copyrighted works. Now go to your favorite model and ask "Give me some code written by Linus Torvalds": not a peep about copyright violation.
... but it does not matter, and it won't matter. Courts are making excuses to allow LLM models to violate any copyright, the excuse does not work, does not convince rational people, but it just doesn't matter.
But of course, if you thought that just because they cheat against the law to make what they're already doing legal, they'll do the same for you, help you violate copyright, right? After all, that's how they work! Ok now go and ask:
"Make me an image of Mickey Mouse peeling a cheese banana under an angry moon"
And you'll get a reply "YOU EVIL COPYRIGHT VILLAIN". Despite, of course, Mickey Mouse no longer being covered under copyright!
And to really get angry, find your favorite indie artist, and ask to make something based on their work. Even "Make an MC Escher style painting of Sonic the Hedgehog" ... even that doesn't count as copyright violation, only the truly gigantic companies deserve copyright protection.
> Given that LLMs were trained on the repository directly, it's not just the case that anything made by the LLM is a derivative work, the LLM ITSELF is a derivative work.
That’s not how “derivative works”, well, work.
First of all, a thing can only be a derivative work if it is itself an original work of authorship.
Otherwise, it might be (or contain) a complete copy or a partial copy of one or more source works (which, if it doesn't fall into a copyright exception, would still be a at least a potential violation), but its not a derivative work.
The LLM would, under that argument, be a transformative derivative work, which has important fair use implications (that don’t exist in the chardet case)…
This is correct. I think any author of a main chunk of code that they claim ownership to (which is probably all of us!) should at least study the basics of copyright law. Getting little details wrong can cost you time, money and eventually your business if you're not careful.
The argument that a rewrite is a copyright violation because they are familiar with the code base is not fully sound.
"Insider Knowledge" is not relevant for copyright law. That is more in the space of patent law then copyright law.
Or else a artist having seen a picture of a sunset over an empty ocean wouldn't be allowed to pain another sunset over an empty ocean as people could claim copyright violation.
Through what is a violation is, if you place the code side by side and try to circumvent copyright law by just rephrasing the exact same code.
This also means that if you give an AI access to a code base and tell it to produce a new code base doing the same (or similar) it will most likely be ruled as copyright violation as it's pretty much a side by side rewriting.
But you very much can rewrite a project under new license even if you have in depth knowledge. IFF you don't have the old project open/look at it while doing so. Rewrite it from scratch. And don't just rewrite the same code from memory, but instead write fully new code producing the same/similar outputs.
Through while doing so is not per-se illegal, it is legally very attackable. As you will have a hard time defending such a rewrite from copyright claims (except if it's internally so completely different that it stops any claims of "being a copy", e.g. you use complete different algorithms, architecture, etc. to produce the same results in a different way).
In the end while technically "legally hard to defend" != "illegal", for companies it's most
times best to treat it the same.
> "Insider Knowledge" is not relevant for copyright law. That is more in the space of patent law then copyright law.
On the contrary. Except for discussions about punitive damages and so on, insider knowledge or lack thereof is completely irrelevant to patent law. If company A has a patent on something, they can assert said patent against company B regardless of whether any person in company B had ever seen or heard of company A and their patent. Company B could have a legal trail proving they invented their product that matches the patent from scratch with no outside knowledge, and that they had been doing this before company A had even filed their patent, and it wouldn't matter at all - company A, by virtue of filing and being granted a patent, has a legal monopoly on that invention.
In contrast, for copyright the right is intrinsically tied to the origin of a work. If you create a digital image that is entirely identical at the pixel level with a copyrighted work, and you can prove that you had never seen that original copyrighted work and you created your image completely independently, then you have not broken anyone's copyright and are free to sell copies of your own work. Even more, you have your own copyright over your own work, and can assert it over anyone that tries to copy your work without permission, despite an identical work existing and being owned by someone else.
Now, purely in principle this would remain true even if you had seen the other work. But in reality, it's impossible to convince any jury that you happened to produce, entirely out of your own creativity, an original work that is identical to a work you had seen before.
> But you very much can rewrite a project under new license even if you have in depth knowledge. IFF you don't have the old project open/look at it while doing so.
No, this is very much false. You will never be able to win a court case on this, as any significant similarity between your work and the original will be considered a copyright violation, per the preponderance of the evidence.
>
In contrast, for copyright the right is intrinsically tied to the origin of a work. If you create a digital image that is entirely identical at the pixel level with a copyrighted work, and you can prove that you had never seen that original copyrighted work and you created your image completely independently, then you have not broken anyone's copyright and are free to sell copies of your own work.
This is not true. I will just give the example of the nighttime illumination of the Eiffel Tower:
If I read Mario Puzo’s The Godfather and then proceed to write a structurally identical novel with many of the same story beats and character types, it will not be difficult to convince a jury exposed to these facts that I’ve created a derivative work.
On the other hand, if I can prove to the jury’s satisfaction that I’ve never been exposed to Puzo’s work in any form, it’s independent creation.
To the contrary, there have been many cases of very similar novels with largely identical plot points and settings that survive copyright allegations, even if the author was exposed to the original work.
In the case of chardet though it wouldn't it be more like you were the publisher of the godfather novel, withdrawing it from print and releasing a novel with the same name with much of the same plot and characters but claiming the new version was an independent creation?
If the new maintainers used Claude as their “fancy code generator” (there’s a Claude.md file in the repository so it seems so) then it was almost certainly trained with the chardet source code.
nobody can tell and this is how we entered this very turbulent modern times of "everything can be retold" without punishment. LLMs already doing it at large, while original author is correct in terms of the LGPL, it is nearly impossible to say how different should expression of an idea be to be considered separate one. this is truly fundamental philosophical question that may not have an easy answer.
Think of a rewrite (by a human or an LLM) as a translation. If you wrote a book in English and somebody translated it into Spanish, it'd still be a copyright issue. Same thing with translations.
That's very different to taking the idea of a body of work. So you can't copyright the idea of a pirate taking a princess hostage and a hero rescuing her. That's too generic. But even here there are limits. There have been lawsuits over artistic works being too similar.
Back to software, you can't copyright the idea of photo-editing software but you can copyright the source code that produces that software. If you can somehow prompt an LLM to produce photo editing software or if a person writes it themselves then you have what's generally referred to as a "cleanroom" implmentation and that's copyright-free (although you may have patent issues, which is a whole separate issue).
But even if you prompted an LLM that way, how did the LLM learn what it needed? Was the source code of another project an input in its training? This is a legal grey area, currently. But I suspect it's going to be a problem.
Suchir Balaji, the OpenAI researcher who was found dead in his flat just before testifying against his employer, published an excellent article somehow related to this topic:
Hate to be "that guy" but in a corrupt legal system, which ours is, none of this matters. Who has the influence and dollars to make the decision theirs is all that matters.
Producing a copy of a copyrighted work through a purely mechanical process is clear violation of copyright. LLMs are absolutely not different from a copier machine in the eyes of the law.
Original works can only be produced by a human being, by definition in copyright law. Any artifact produced by an animal, a mechanical process, a machine, a natural phenomenon etc is either a derived work if it started from an original copyrighted work, or a public domain artifact not covered by copyright law if it didn't.
For example, an image created on a rock struck by lightning is not a copyright covered work. Similarly, an image generated by an diffusion model from a randomly generated sentence is not a copyrightable work. However, if you feed a novel as a prompt to an LLM and ask for a summary, the resulting summary is a derived work of said novel, and it falls under the copyright of the novel's owner - you are not allowed to distribute copies of the summary the LLM generated for you.
Whether the output of an LLM, or the LLM weights themselves, might be considered derived works of the training set of that LLM is a completely different discussion, and one that has not yet been settled in court.
Perhaps - but an argument might still be made that the result is a derivative work of the original, given that it's produced by feeding the original work through automated tooling.
But either way, deleting the original version from the repo and replacing it with the new version - as opposed to, say, archiving the old version and starting a new repo with the new version - would still be a dick move.
As part of my consulting, i've stumbled upon this issue in a commercial context.
A SaaS company who has the mobile apps of their platform open source approached me with the following concern.
One of their engineers was able to recreate their platform by letting Claude Code reverse engineer their Apps and the Web-Frontend, creating an API-compatible backend that is functionally identical.
Took him a week after work. It's not as stable, the unit-tests need more work, the code has some unnecessary duplication, hosting isn't fully figured out, but the end-to-end test-harness is even more stable than their own.
"How do we protect ourselves against a competitor doing this?"
You're not describing anything new, you're describing progress. A company invests time and money and expertise into building a product, it becomes established, people copy in 1/10th of the time, the quality of products across the industry improve. Long before generative AI, Instagram famously copied Snapchat's stories concept in a weekend, and that is now a multi-multi-multi-billion contributor to Meta's bottom line.
As engineers, we often think only about code, but code has never been what makes a business succeed. If your client thinks that their businesses primary value is in the mobile app code they wrote, 1) why is it even open source? 2) the business is doomed.
Realistically, though, this is inconsequential, and any time spent worrying about this is wasted time. You don't protect yourself from your competitor by worrying about them copying your mobile app.
They do something very similar for some of their work. It’s hard to use external services so they replicate them and the cost of doing so has come down from “don’t be daft, we can’t reimplement slack and google drive this sprint just to make testing faster” to realistic. They run the sdks against the live services and their own implementations until they don’t see behaviour differences. Now they have a fast slack and drive and more (that do everything they need for their testing) accelerating other work. I’m dramatically shifting my concept of what’s expensive and not for development. What you’re describing could have been done by someone before, but the difficulty of building that backend has dropped enormously. Even if the application was closed you could probably either now or soon start to do the same thing starting with building back to core user stories and building the app as well.
You can view some of this as having things like the application as a very precise specification.
I think it's interesting to add what they use it for and why its hard.
What they use it for:
- It's about automated testing against third party services.
- It's not about replicating the product for end users
Why using external services is hard/problematic
- Performance: They want to have super fast feedback cycles in the agentic loop: In-Memory tests. So they let the AI write full in-memory simulations of (for example) the slack api that are behaviorally equivalent for their use cases.
- Feasiblity: The sandboxes offered by these services usually have performance limits (= number of requests per month, etc) that would easily be exhausted if attached to a test harness that runs every other minute in an automated BDD loop.
> "How do we protect ourselves against a competitor doing this?"
If the platform is so trivial that it can be reverse engineered by an AI agent from a dumb frontend, what's there to protect against? One has to assume that their moat is not that part of the backend but something else entirely about how the service is being provided.
Interesting case, IANAL but sounds legal and legit. The AI did not have expose to the backend it re-implemented. The API itself is public and not protectable.
No serious enterprise SaaS company differentiates themselves solely on the product (the products are usually terrible). It's the sales channel, the fact that you know how to bill a big company, the human engineer who is sent on site to deploy and integrate the product, the people on the support line 24/7, the regulatory framework that ensures the customer can operate legally and obtain insurance, the fact that there's a deep pool of potential hires who have used and understand the product. Those are the differentiators.
> "How do we protect ourselves against a competitor doing this?"
You can try patenting; but not after the fact. Copyright won't help you here. You can't copyright an algorithm or idea, just a specific form or implementation of it. And there is a lot of legal history about what is and isn't a derivative work here. Some companies try to forbid reverse engineering in their licensing. But of course that might be a bit hard to enforce, or prove. And it doesn't work for OSS stuff in any case.
Stuff like this has been common practice in the industry for decades. Most good software ideas get picked apart, copied and re-implemented. IBM's bios for the first PC quickly got reverse engineered and then other companies started making IBM compatible PCs. IBM never open sourced their bios and they probably did not intend for that to happen. But that didn't matter. Likewise there were several PC compatible DOS variants that each could (mostly) run the same applications. MS never open sourced DOS either. There are countless examples of people figuring out how stuff works and then creating independent implementations. All that is perfectly legal.
Between this and the fact that their PC-DOS (née MS-DOS) license was nonexclusive, I'm honestly not sure what they expected to happen.
The nature of early IBM PC advertising suggests to me that they expected the IBM name and established business relationships to carry as much weight as the specifications itself, and that "IBM PC compatible" systems would be no more attractive than existing personal computers running similar if not identical third-party software (PC-DOS wasn't the only example of IBM reselling third-party software under nonexclusive license), and would perhaps even lead to increased sales of first-party IBM PCs.
Which, in fact, they did, leading me to believe the actual result may have been not too far from their original intent, only with IBM capturing and holding a larger share of the pie.
> "How do we protect ourselves against a competitor doing this?"
I have been thinking about this a lot lately, as someone launching a niche b2b SaaS. The unfortunate conclusion that I have come to is: have more capital than anyone for distribution.
Is there any other answer to this? I hope so, as we are not in the well-capitalized category, but we have friendly user traction. I think the only possible way to succeed is to quietly secure some big contracts.
I had been hoping to bootstrap, but how can we in this new "code is cheap" world? I know it's always been like this, but it is even worse now, isn't it?
I think the genie is out of the bottle on this one and there's really no putting it back.
There is a certain amount of brand loyalty and platform inertia that will keep people. Also, as you point out, just having the source code isn't enough. Running a platform is more than that. But that gap will narrow with time.
The broader issue here is that there are people in tech who don't realize that AI is coming for their jobs (and companies) too. I hope people in this position can maybe understand the overall societal issues for other people seeing their industries "disrupted" (ie destroyed) by AI.
The copyright argument is a sidetrack both in the PR comment thread and here. The issue opened claims the new code is based on the old code, and therefore derivative, and therefore must be offered in a modified version of the source code under the previous license, LGPL. The complaint is the maintainers violated the terms of LGPL, that they must prove no derivation from the original code to legally claim this is a legal new version without the LGPL license. Claim is if they or Claude read the old code (or of course directly use any of it) it is a license violation. “… in the release 7.0.0, the maintainers claim to have the right to “relicense” the project. They have no such right; doing so is an explicit violation of the LGPL. Licensed code, when modified, must be released under the same LGPL license. Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code (i.e. this is not a "clean room" implementation).“ By this reasoning, I am genuinely asking (I’m not a license expert) if a valid clean room rewrite is possible, because at a minimum you would need a spec describing all behavior, which ses to require ample exposure to the original to be sufficiently precise.
That's not what a derivative work means, though. Being exposed to something doesn't mean you can't create original work which is similar to it (otherwise every song or artwork would be a derivative work of everything before it)
People do cleanroom implementations as a precaution against a lawsuit, but it's not a necessary element.
In fact, even if some parts are similar, it's still not a clear-cut case - the defendant can very well argue that the usage was 1. transformative 2. insubstantial to the entirety of work.
"The complaint is the maintainers violated the terms of LGPL, that they must prove no derivation from the original code to legally claim this is a legal new version without the LGPL license."
The burden of proof is on the accuser.
"I am genuinely asking (I’m not a license expert) if a valid clean room rewrite is possible, because at a minimum you would need a spec describing all behavior, which ses to require ample exposure to the original to be sufficiently precise."
Linux would be illegal if so (they had knowledge of Unix before), and many GNU tools are libre API-compatible reimplementations of previous Unix utilities :)
The copyright argument is the only relevant argument. If the new work is a derived work of the original, then it follows by definition that the new work is under the copryight of the original's author(s). Since the original chardet was distributed by its author(s) only under the LGPL, any copy/derivative of it that anyone else creates must be distributed only under the LGPL, per the terms of the LGPL.
Now, whether chardet 7.0.0 is a derivative of chardet or not is a matter of copyright law that the LGPL has no say on, and a rather murky ground with not that much case law to rely on behind it. If it's not, the new author is free to distribute chardet 7.0.0 under any license they want, since it is a new work under his copyright.
> Claim is if they or Claude read the old code (or of course directly use any of it) it is a license violation
The original code is part of claude's training material. With that intepretation of the LGPL AI is incapable of writing non LGPL derivatives. I like that interpretation.
I'm not certain I buy it, but I find it a little hard to motivate the training being fair use if used to regenerate the project in a different licence.
Maintainers must not be able to change the license that original author chose, and based on which contributors made contributions. When one stepped up to be maintainer, it was a trustee role, not owner role.
It should be perfectly ok (by maintainer or anyone for that mater) to be inspired from a community project and build something from scratch hand-crafting/ AI sloping, as long as the imitation is given a new name/ identity.
What rubbed me off personally was maintainer saying "pin your dependncies to version 6.0.0 or 5.x.x", as if maintainer owns the project. maintainer role is more akin to serve the community, not rule.
If it is completely new, why not start a new project with new name? No one will object. And of course leave the old project behind to whoever is willing to maintain it.
And if the new name project is better, people will follow.
A distinction should be made between ownership of the code and its copyright and ownership of the repository and associated distribution channels. As far as I know, there's no precedent to state that owning the former means ownership of the latter. The original author abandoned this project years ago and likely has no legal basis to maintain that the project itself stays LGPL, only that their code and derivatives of it stay LGPL. Unless it could be proven otherwise, a rewrite of the project under a different license made without directly referencing the original is likely well within the rights of the owners of the repository to do.
Say a community builds a hall with an explicit intent of community use only, led by single person or a group, and then a person/ a group is appointed as a caretaker. Caretaker of the community hall decides unilaterally to convert the hall into a business convention center, razing old building to ground and rebuilding, disregarding community wishes, to make hall business friendly. How would you react to this situation? Caretaker has the ownership of the community assets, including the ground on which the hall is standing?
my understanding of the situation is:
Is the caretaker paying from his own pocket to maintain the hall? no
Is the caretaker paying from his own pocket for community usage of the hall? no
Is the caretaker spending time to maintain the community hall? yes
Is caretaker obliged to spend time on community hall? no
Is caretaker free to stop spending time on community hall? yes.
Is caretaker free to raze current hall, build new hall on same ground for new purposes WITH community agreement? YES
Is caretaker free to raze current hall, build new hall on same ground for new purposes WITHOUT community agreement (even if paying all the bill)? NO
Is caretaker free to build another similar hall someplace else? YES
Reasoning of your comment is of someone who is hell bent on staking claims on community resources (like big companies) without having slightest concern of the wishes or well-being of the community. Not sure of the commenter's motive either, given the new account with just two comments, supporting such blatant disregard of basic human decency.
Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code.
Question: if they had built one using AI teams in both “rooms”, one writing a spec the other implementing, would that be fine? You’d need to verify spec doesn’t include source code, but that’s easy enough.
It seems to mostly follow the IBM-era precedent. However, since the model probably had the original code in its training data, maybe not? Maybe valid for closed source project but not open-source? Interesting question.
> Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code.
It doesn't matter how they structure the agents. Since chardet is in the LLM training set, you can't claim any AI implementation thereof is clean room.
Might still be valid for closed source projects (probably is).
I think courts would need to weigh in on the open source side. There’s legal precedent is that you can use a derived work to generate a new unique work (the spec derived for the copyrighted code is very much a derived work). There are rulings that LLMs are transformative works, not just copies of training data.
LLMs can’t reproduce their entire training set. But this thinking is also ripe for misuse. I could always train or fine-tune a model on the original work so that it can reproduce the original. We quickly get into statistical arguments here.
So by that logic, you're not legally allowed to implement your own character detector and license it as your own if you've ever looked at chardet's source code? I'm confused. I thought copyright laws protect intellectual property as-is, not the impression it leaves on someone.
This seems right to me. If you ask a LLM to derive a spec that has no expressive element of the original code (a clean-room human team can carefully verify this), and then ask another instance of the LLM (with fresh context) to write out code from the spec, how is that different from a "clean room" rewrite? The agent that writes the new code only ever sees the spec, and by assumption (the assumption that's made in all clean room rewrites) the spec is purely factual with all copyrightable expression having been distilled out. But the "deriving the spec (and verifying that it's as clean as possible)" is crucial and cannot be skipped!
How would a team verify this for any current model? They would have to observe and control all training data. In practice, any currently available model that is good enough to perform this task likely fails the clean room criteria due to having a copy of the source code of the project it wants to rewrite. At that point it's basically an expensive lossy copy paste.
While it feels unlikely that a simple "write this spec from this code" + "write this code from this spec" loop would actually trigger this kind of hiding behaviour, an LLM trained to accurately reproduce code from such a loop definitely would be capable of hiding code details within the spec - and you can't reasonably prove that the frontier LLMs have not been trained to do so.
Yeah I think, the Compaq / IBM precedent can only superficially apply. It would be like having two teams only meet in a room full of documentation - but both teams crammed the source code the day before. (That, the source code you are "reverse engineering" is in the training data.) It doesn't make sense.
Also, it's weird that it's okay apparently to use pirated materials to teach an LLM, but maybe not to disseminate what the LLM then tells you.
> Unfortunately, because the code that chardet was originally based on was LGPL, we don't really have a way to relicense it. Believe me, if we could, I would. There was talk of chardet being added to the standard library, and that was deemed impossible because of being unable to change the license.
So the person that did the rewrite knew this was a dive into dangerous water.
That's so disrespectful.
> Licensed code, when modified, must be released under the same LGPL license. Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code (i.e. this is not a "clean room" implementation).
I don't think that the second sentence is a valid claim per se, it depends on what this "rewritten code" actually looks like (IANAL).
Edit: my understanding of "clean room implementation" is that it is a good defence to a copyright infrigement claim because there cannot be infringement if you don't know the original work. However it does not mean that NOT "clean room implementation" implies infrigement, it's just that it is potentially harder to defend against a claim if the original work was known.
I agree that (while the ethics of this are a different issue) the copyright question is not obviously clear-cut. Though IANAL.
As the LGPL says:
> A "work based on the Library" means either the Library or any derivative work under copyright law: that is to say, a work containing the Library or a portion of it, either verbatim or with modifications and/or translated straightforwardly into another language. (Hereinafter, translation is included without limitation in the term "modification".)
Is v7.0.0 a [derivative work](https://en.wikipedia.org/wiki/Derivative_work)? It seems to depend on the details of the source code (implementing the same API is not copyright infringement).
I was wondering how the existing case law of translated works, from one language to an other works here. It would at suggest that this is an infringement of the license especially because of the lack of creativity. But IANAL and of course no idea of applicable case law.
the ai copy pasted the existing project. How can such a procedure not fall under copyright?
Especially now that ai can do this for any kind of intellectual property, like images, books or sourcecode. If judges would allow an ai rewrite to count as an original creation, copyright as we know it completely ends world wide.
Instead whats more likely is that no one is gonna buy that shit
I came here to say this. While I agree with Mark that what they’re doing is not nice, I’m not sure it’s wrong. A clean-room implementation is one way the industry worked around licensing in the past (and present, I guess), but it’s not a requirement in law as far as I know.
I’m not sure that “a total rewrite” wouldn’t, in fact, pass muster - depending on how much of a rewrite it was of course. The ‘clean room’ approach was just invented as a plausible-sounding story to head off gratuitous lawsuits. This doesn’t look as defensible against the threat of a lawsuit, but it doesn’t mean it wouldn’t win that lawsuit (I’m not saying it would, I haven’t read or compared the code vs its original). Google copied the entire API of the Java language, and got away with it when Oracle sued. Things in a courtroom can often go in surprising ways…
[edit: negative votes, huh, that’s a first for a while… looks like Reddit/Slashdot-style “downvote if you don’t like what is being said” is alive and well on HN]
I spent like two minutes looking at the diff between the original and the supposed "clean room" implementation [1] and already found identical classes, variable names, methods, and parameters. It looks like there was no actual attempt at clean-rooming this, regardless of whether that "counts".
Lol at the statement that "clean room" would have been invented to scare people from suing. It's the opposite: clean room is a fairly-desperate attempt to pre-empt accusations in court when it is expected that the "derivative" argument will be very strong, in order to then piggyback on the doctrine about interoperability. Sometimes it works, but it's a very high bar to clear.
It will hold up in court. The line of argument of “well I went into a dark room with only the first Harry Potter book and a type writer and reproduced the entire work, so now I own the rewrite” doesn’t hold up in court, it doesn’t either when when you put AI in the mix. It doesn’t matter if the result is slightly different, a judge will rule based on the fact that this even is literally what the law is intended to prevent, it’s not a case of which incantation or secret sentence you should utter to free the work of its existing license.
> “well I went into a dark room with only the first Harry Potter book and a type writer and reproduced the entire work, so now I own the rewrite”
This is not a good analogy.
A "rewrite" in context here is not a reproduction of the original work but a different work that is functionally equivalent, or at least that is the claim.
There's a subtext in your point that I want to expand on.
Tech people, particularly engineers, tend to make a fundamental error when dealing with the law that almost always causes them to make wrong conclusions. And that error is that they look for technical compliance when so much of the law is subjective and holistic.
An example I like to use is people who do something illegal on the Internet and then use the argument "you can't prove I did it (with absolute certainty)". It could've been someone who hacked your Wifi. You don't know who on the Wifi did it, etc. But the law will look at the totality of the evidence. Did the activity occur when you were at home and stop when you weren't? How likely are alternative explanations? Etc.
All of that will be considered based on some legal standard depending on the venue. In civil court that tends to be "the preponderance of the evidence" (meaning more likely than not) while in criminal court it's "beyond a reasonable doubt" (which is a much higher standard).
So, using your example, an engineer will often fall into a trap of thinking they can substitute enough words to have a new original work, Ship of Theseus-like. And the law simply doesn't work that way.
So, when this gets to a court (which it will, it's not a question of "if"), the court will consider how necessary the source work was to what you did. If you used it for a direct translation (eg from C++ to Go) then you're going to lose. My prediction is that even using it in training data will be cause for a copyright claim.
If you use Moby Dick in your training data and ask an LLM to write a book like Moby Dick (either explicitly or implicitly) then you're going to have an issue. Even if you split responsibilities so one LLM (training on Moby Dick) comes up with a structure/prompt and another LLM (not trained on Moby Dick) writes it, I don't think that'll really help you avoid the issue.
> So, when this gets to a court (which it will, it's not a question of "if"), the court will consider how necessary the source work was to what you did. If you used it for a direct translation (eg from C++ to Go) then you're going to lose. My prediction is that even using it in training data will be cause for a copyright claim.
This has a lot of similarity to when colorization of film started popping up. Did colorizing black and white movies suddenly change the copyright of the film? At this point is seems mostly the courts say no. But you may find sometimes people rule the other way and say yes. But it takes time and a lot of effort to get what in general people want.
But basically if you start with a 'spec' then make something you probably can get a wholly owned new thing. But if you start with the old thing and just transform it in some way. You can do that. But the original copyright holders still have rights too to the thing you mangled too.
If I remember right they called it 'color of copyright' or something like that.
The LLM bits you are probably right. But that has not been worked out by the law or the courts yet. So the courts may make up new case law around it. Or the lawmakers might get ahead of it and say something (unlikely).
> And that error is that they look for technical compliance when so much of the law is subjective and holistic.
I know it sounds like an oversimplification, but "got off on a technicality" is a common thing among the well-connected and well-heeled. Sure, us nerds probably focus too much on the "technicality" part, since we are by definition technical, but the rest is wishy-washy, unfair BS as far as many of our brains work much of the time.
Everyone armchair debating the licensing side of this frustrates discussion, because the only licensing discussion that matters is the one in front of a court. Until and unless one happens, this is just a boring hobby.
Otherwise all this rewrite accomplishes is a 2.3% accuracy improvement and some performance gains that might not be relevant in production, in exchange for a broken test suite, breaking changes, and unnecessary legal and ethical risks pushed out as an update to what was already a stable project.
If it's truly a sufficiently separate project that it can be relicensed from LGPL, then it could've just been _a fully separate project with a new identity_, and the license change would've been at least harder to challenge. Instead, we're here.
Being completely untainted is the standard many reimplementations set for themselves to completely rule out legal trouble. For example ReactOS won't let you contribute if you have ever seen Windows code. Because if you have never seen it, there can be no allegation that you copied it.
That is however stricter than what's actually legally necessary. It's just that the actual legal standard would require a court ruling to determine if you passed it, and everyone wants to avoid that. As a consequence there also aren't a lot of court cases to draw similarities to
> For example ReactOS won't let you contribute if you have ever seen Windows code. Because if you have never seen it, there can be no allegation that you copied it.
I've heard this called in some circles "The curse of knowledge." The same thing applies to emulator developers, especially N64 developers (and now Nintendo emulator developers in general) after the Oman Archive and later Gigaleaks. There's an informal "If you read this, you can NEVER directly contribute to the development of that emulator, ever."
This comes to a head when a relatively unknown developer starts contributing oddly specific patches to an emulator.
"Taint" requires that the code is demonstratably derivative from the *GPL licensed work.
This is actually harder standard than some people think.
The absolute clean room approaches in USA are there because they help short circuit a long lawsuit where a bigger corp can drag forever until you're broken.
It's harder than some people think, but the author does a lot of the work when he names the resulting artifact "chardet v7.0.0". If I thought I was writing the kind of arms-length reimplementation that's required, I would never put it into the versioning scheme of the original, come on.
Not a lawyer, but that always seemed naively correct to me.
However, the copyright system has always be a sham to protect US capital interests. So I would be very surprised if this is actually ruled/enforced. And in any case american legislators can just change the law.
If the code is different but API compatible, Google Java vs Oracle Java case shows that if the implementation is different enough, it can be considered a new implementation. Clean room or not.
That whole clean room argument makes no sense. Project changed governance and was significantly refactored or reimplemented... I think the maintainers deserve to call it their own. Original-pre MIT release can stay LGPL.
I don't think this is a precedent either, plenty of projects changed licenses lol.
I keep kind mixing them up but the GPL licenses keep popping up as occasionally horror stories. Maybe the license is just poorly written for today's standards?
Governance change or refactoring don’t give you a right to relicense someone else’s work. It needs to be a whole new work, which you own the copyright to.
It's not clear at all why the current maintainers wanted/needed this re-licensing. I guess that their employee, Monarch Money, wants to use derivative work in their application without releasing the changes?
It was already LGPL, perfect for a library, not GPL.
That take and the link within it to https://github.com/chardet/chardet/issues/36 which is from 2014 (!!) and revived in 2021 (!) make me think now that Dan Blanchard is in good faith, but still acted in a naive manner. Rewriting from "scratch" with Claude Code and basically taking over the project by changing the license, all in one single pull request, well... it's not going to end well on the public relations side.
Copyright protects "authorship", not functionality. Patent protects functional elements.
A rewrite based on functional equivalency is not infringing on the copyright as long as no creative expression was copied. That was the heart of the Google case, whether the API itself was creative expression or functionality.
There are many aspects to what can be considered creative expression, including names, organization, non-functional aspects. An algorithm would not be protected expression.
If an AI can write it without reference to the original source code, using only documented behavior, then it would not be infringing (proving that it didn't copy anything from training data might be tough though). It also would not itself be copyrightable, except for elements that could be traced back as "authorship" to the humans who worked with the AI.
If LLMs can create GOOD software based only on functionality, not by copying expression, then they could reproduce every piece of GPL software and release it as Public Domain (which it would have to be if no human has any authorship in it). By the same principle that the GPL software wasn't infringing on the programs they copied functionality from, neither would the AI software.
That's a big IF at this point, though, the part about producing GOOD software without copying.
Look, forget the details, step back and consider the implications of the principle.
Someone should not be able to write a semi-common core utility, provide it as a public good, abandon it for over a decade, and yet continue to hold the rest of the world hostage just because of provenance. That’s a trap and it’s not in any public interest.
The true value of these things only comes from use. The extreme positions for ideals might be nice at times, but for example we still don’t have public access to printer firmware. Most of this ideology has failed in key originating goals and continues to cause headaches.
If we’re going to share, share.
If you don’t want to share, don’t.
But let’s not setup terminal traps, no one benefits from that.
If we flip this back around though, shouldn’t this all be MPL and Netscape communications? (Edit: turns out they had an argument about that in the past on their own issue tracker: https://github.com/chardet/chardet/issues/36)
LGPL applies to the LGPL’d code, not to every piece of code someone might add to the repository or under the same name implicitly.
The claim being made is that because some prior implementation was licensed one way, all other implementations must also be licensed as such.
AIUI the code has provenance in Netscape, prior to the chardet library, and the Netscape code has provenance in academic literature.
Now the question of what constitutes a rewrite is complex, and maybe somewhat more complex with the AI involvement, but if we take the current maintainers story as honest they almost certainly passed the bar of independence for the code.
I've got some serious moral questions about "rewrite your own widely used project from scratch with the same package name"... but I don't think it's fair to call this "someone else's project" when the OP has apparently been the only maintainer working on the project for 13 years...
Isn't the real issue here that tons of projects that depend on the "chardet" now drag in some crappy still unverified AI slop? AI forgery poisoning, IMHO.
Why does this new project here needed to replace the original like that in this dishonourable way? The proper way would have been to create a proper new project.
Note: even Python's own pip drags this in as dependency it seems (hopefully they'll stick to a proper version)
This indeed the real issue (not the AI angle per se, but the wholesale replacement. The licensing issue is real, but less important IMO).
Half a million lines of code have been deleted and replaced over the course of four days, directly to the main branch with no opportunity for community review and testing. (I've no idea whether depending projects use main or the stable branch, but stable is nearly 4 years old at this point, so while I hope it's the version depending projects use, I wouldn't put money on it.)
The whole thing smells a lot like a supply chain attack - and even if it's in good faith, that's one hell of a lot of code to be reviewed in order to make sure.
Woah. As someone not in this particular community but dependent on these tools this is exactly the terrifying underbelly we've all discussed with the user architecture of tools like pip and npm. It's horrifying that a major component just got torn apart, rebuilt, and deployed to anyone who uses those python ecosystems (... many millions? ... billions of people?)
The test coverage is going to be entirely different, unless of course they copied the tests, which would then preclude them from changing the license. They didn't even bother to make sure the CI passed on merging a major version release https://github.com/chardet/chardet/actions/runs/22563903687/...
The drop"-in" compatibility claims are also just wrong? I ran it on the old test suite from 6.0 (which is completely absent now), and quickly checking:
- the outputs, even if correctly deduced, are often incompatible: "utf-16be" turns into "utf-16-be", "UTF-16" turns into "utf-16-le" etc. FWIW, the old version appears to have been a bit of a mess (having had "UTF-16", "utf-16be" and "utf-16le" among its outputs) but I still wouldn't call the new version _compatible_,
- similarly, all `ascii` turn into `Windows-1252`
- sometimes it really does appear more accurate,
- but sometimes it appears to flip between wider families of closely related encodings, like one SHIFT_JIS test (confidence 0.99) turns into cp932 (confidence 0.34), or the whole family of tests that were determined as gb18030 (chinese) are now sometimes determined as gb2312 (the older subset of gb18030), and one even as cp1006, which AFAIK is just wrong.
As for performance claims, they appear not entirely false - analyzing all files took 20s, versus 150s with v6.0. However, looks like the library sometimes takes 2s to lazy initialize something, which means that if one uses `chardetect` CLI instead of Python API, you'll pay this cost each time and get several times slower instead.
Oh, and this "Negligible import memory (96 B)" is just silly and obviously wrong.
Impossible to code inspect and verify. Someone else commented 'smells like a supply chain attack' and while I'm sure it's not intended to be, there is no way to verify. And who believes, in half a million lines of code, no bugs have been introduced?
AFAIK this was not a clean room reimplementation. But since it was rewritten by hand, into a different language, with not just a different internal design but a different API, I could easily buy that chardetng doesn't infringe while Python chardet 7 does.
You have to look from two sides this
Moral: What is right or wrong? If they wanted to change the license, they could have made another project with another name, and nobody would care, but they wanted the reputation of the project.
Legal: How much are you willing to spend on litigation? The only real "protection" by copyright is in court.
Further, while the copyright of the original code and its derivatives are still owned by the original author does that hold true to the rights to the name and package namespace? Supposing this were indisputably a clean room implementation instead of an unclear one, would the maintainers then have a right to relicense under the same name? I would imagine that yes, they do have the right to relicense in that case because the copyright only applies to the code, not the project itself.
Other questions that haven't really been explored before also are maintained: the original author hasn't been involved in some time, technically the copyright of all code since still belongs to those authors who might be bound by LGPL but are also the only ones with the right to enforce it and could simply choose not to. What then?
I wonder if LLMs will push the industry towards protecting their IP with patents like the other branches of engineering rather than copyright. If you patent a general idea of how your software works then no rewrite will be able to lift this protection.
If there wasn't a specific limitation against software patents, you could patent the process of "first we look at the first 3 bytes for a BOM, then we take statistics of the most frequently used bytes and match them against this table..." which is the software equivalent of "the device contains a large box full of water, underneath which is a flame and on the upper side is a pipe leading to a valve box which contains..."
Either one would still have to meet the requirements like being sufficiently non-obvious. The first steam engine was patented, even though you couldn't patent one any more.
Does anyone understand the intent behind the changed license on the package, why are the current maintainers trying to do it in the first place? What's actually going on?
Unicode detection is the kind of utility the language maintainers want in their package collection if not in the standard library, and programmers who have to do anything with "plain text" files might want to rely on.
Releasing a core library like this under a genuinely free licence (MIT) is a service to anyone working in the ecosystem.
Nobody that is not already writing (L)GPL licensed software wants to pull LGPL licensed libraries into his software. I'm not a lawyer. If you have it in separate object artefacts that you can dynamically link to, you should be fine. Everything else may be more difficult.
Interestingly the original author ported the code from the Mozilla C++ version which is licensed as "MPL 1.1 or LGPL" (it's a bit unclear as the readme says that but the license file mentions only MPL). So the author did already relicence the project in a way by licensing the port as LGPL only.
I am confused. In the USA, there has been a clear rule that machine-generated code cannot be copyright. If the "new implementation" was in fact created by Claude (which is my impression), then nobody holds any copyright to the code, and it cannot be licensed under any license at all.
This is effectively a contract. You can put anything you want in a contract, but contracts are enforceable to only to the extent they comply with the law (statutes, case law, the constitution, etc.)
So to settle this, someone needs to violate this license and get sued. Or maybe proactively sue?
Remember: this applies to all LLM-generated code, not just chardet. No LLM-generated code is copyrightable, and thus cannot be licensed. The legal challenge could come in any context where LLMs have been used and the code placed under any license (proprietary or otherwise).
Which is going cause a collision between the "not copyrightable" and "derived from copyrighted work" angles.
The machine cannot get authorship, but just as images created by humans with PhotoShop and all kinds of machinery are still copyrighted to the human creator - unless they explicitly set up some circumstances where the process of creation happens completely without them - code/software produced by a machine instructed by a human should get copyright (either original or derivative).
Unless the human is so far removed from the output. (And how far is far enough is probably very much depends on the circumstances and unless/until case law or Congress gives us some unifying criteria, it's going to be up to how the judge and the jury feels.)
..
For example someone set up a system where their dog ends up promoting some AI to make video games. This might be the closest to the case of that photo.
Though there the court ruled only that PETA (as friend of the monkey) cannot sue the photographer, because the monkey cannot be a copyright holder, but very importantly it didn't rule on the authorship of the photographer. (And thus the Wikimedia metadata which states that the image is in thr public domain, is simply their opinion.)
Setting aside the legal questions, what a nasty thing to do. I would expect to see this kind of move from some big corpo, not an OSS maintainer. This feels like it runs counter to the whole open source ethos and undermines the idea that authorship means anything anymore.
AI "translation" kills any incentive to open source code.
What's worse - disassembler+AI is good enough to "translate" the binary into working source code, probably in a different programming language than then original.
I finally had to mute r/isthisai on Reddit because there’s now a subset of people who see the hand of AI in everything. Could that be generated by a clanker? Sure, but it’s also exactly what I would write if I wanted a quick pitch for a library that addresses some immediate concerns. It’s also what I would focus on if the fact we had just finished a rebuild from scratch.
As Freud famously said, sometimes an em dash is just an em dash.
FWIW, I don't think there's even a room for interpretation here, given the commit that created the README (and almost all commits since the rewrite started 4 days ago) is authored by
We are both trying to steer an internet towards a place we want to be.
The em dash is just a bonus, the grammatical structure is the giveaway. I'd invite Blanchard to argue that it wasn't LLM-generated.
I use AI tooling all day every day and can easily pick out when something was written by most popular modern models. I welcome an agentic web; it's the inevitable future. But not like this. I want things to get better, not worse.
For me, some projects I start by writing a readme.txt by hand. That saves me time in cases I realize I'd be making something pointless. (I don't use chatbots when coding though)
The law is what a court says it is; there is precedent for decisions on human rewrites but LLM (assisted) code might still be fairly uncharted territory.
I think Mark Pilgrim misrepresents the legal situation somewhat: The AI rewrite does not legally need to be a clean room implementation (whatever exactly that would even mean here).
That is just the easiest way to disambiguate the legal situation (i.e. the most reliable approach to prevent it from being considered a derivative work by a court).
Another tangent that I didn't see in the thread is that the Supreme Court just confirmed a ruling that LLM created art isn't copyrightable since the author must be human for copyright to apply.
If the new code was generated entirely by an LLM, can it be licensed at all? Or is it automatically in the public domain?
The US supreme court declined to hear a challenge to the claim that if you explicitly disclaim any human input to make a point then the art isn't copyrightable in the US. The copyrightability of "actually I had some design input" is still up in the air in the US, and copyrightability in general is still up in the air in probably the entire rest of the world as well as every court in the US outside of the DC circuit (because the supreme court declining to hear a case does not constitute an endorsement of the lower courts ruling or create precedent).
There's absolutely nothing stopping you granting a license to public domain work... granting a license is just waiving rights that the author might have to sue for copyright infringement under certain circumstances...
Personally I'd be unwilling to use this work without the license, because I would not be confident that it was public domain.
It would be in the public domain. Wouldn't really matter all that much if the end goal was to get it included in the Python standard library, but the whole "Copyright (c) 2024 Dan Blanchard" in the license file would just be BS.
The big question is whether or not is it a derivative work of an LGPL project. If it is, then it's just an outright copyright violation.
Another day in the post-copyright world. Surely someone somewhere is already using this to test the effect of copyright laws, should we decide to go back to that world.
I think it's just the GPL family of licenses that tend tend to cause most problems. I appreciate their intent, but the outcome often leaves a lot to be desired.
The GPL exists for the benefit of end users, not developers. It being a chore for developers who want to deny their users the software freedoms is a feature, not a bug.
If you have ill intentions or maybe you're a corporation that wants to use someone else's work for free without contributing anything back, then yes, I can see how GPL licenses "tend to cause problems".
Why? What's your problem with them? They do exactly what they're supposed to do, to ensure that future derivatives of the source code have to be distributed under the same license and distribution respects fundamental freedoms.
I like to think about GPL as a kind of an artistic performance and an elaborate critique of the whole concept of copyright.
Like, "we don't like copyright, but since you insist on enforcing it and we can't do anything against it, we will invent a clever way to use your own rules against you".
Wtf are these comments? A LGPL licensed project, guaranteed to be free and open source, being LLM-washed to a permissive license, and GPL is the problem here?
They are literally stealing from open source, but it's the original license that is the issue?
The 25519 crypto package that's built into practically everything these days (SSH, TLS, most e2e messaging) was released as both a spec and a C reference implementation _in the public domain_.
While I am obviously Team GPL and not team "I 'rewrote' this with AI so now it's mine", I'm team anti-fork, and definitely not team 'Chardet'.
Forking should be a last resort, one better option is to yeet the thing entirely.
And chardet lends itself perfectly for this, using chardet is a sign of an issue and low craftmanship, either by the developer using chardet, or the developer that failed to signal the encoding of their text. (See Joel Spolsky's "The absolute minimum every developer should know about character encoding") (And let's be honest, it's probably the developers problem, not everything is someone else's fault.)
Just uninstall this thing where you can, and avoid installing it always, because you always can.
You know I'm right. I will not be replying to copium
I feel like the author is missing a huge point here by fighting this. The entire reason why GPL and any other copyleft license exists in the first place is to ensure that the rights of a user to modify, etc a work cannot be ever taken away. Before, relicensing as MIT - or any other fully permissive license - would've meant open doors to apply restrictions going forward, but with AI this is now a non-issue. Code is now very cheap. So the way I see this, anyone who is for copyleft should be embracing AI-created things as not being copyrightable (or a rewrite being relicensable) hard*.
The user is the end-user of the product. If the relicensing means that someone down the line receives a close-down binary application that he cannot modify, that's a violation of the user's rights.
But it's a non-issue as said user can just have AI reverse engineer said binary. Or reimplement something with the same specs. That's what it means for code to be cheap.
That is still true, but it was more relevant back when "user" meant "programmer at another university". The "end-user" for most software is not a programmer these days.
If I release blub 1.0.0 under GPL, you cannot fork it and add features and release that closed-source, but I can certainly do that as I have ownership. I can't stop others continuing to use 1.0.0 and develop it further under the GPL, but what happens to my own 1.1.0 onwards is up to me. I can even sell the rights to use it closed-source.
You can do whatever you want. And someone else can - and will, if it's worth it - now do a deep analysis and have it reimplemented with said analysis, all via LLM. And particularly, since the law is now that AI creations can't be copyrighted, that reimplementation will be firmly in public domain. Nobody will have to even look at 1.1.0. See Valkey as one of the more recent examples, and there isn't even any LLM involved there.
Well put. It seems like a lot of people have tunnel vision on this. It's going to take a while for people to realize that copyright is obsolete, especially in its current form.
Isn’t it? I mean 12 stage pipeline has a very specific meaning to me in this area, and is not a new way of describing something. The release notes description sounds like a multi stage pipeline.
Do you know this kind of area and are commenting on the code?
I believe that Pilgrim here does not understand very well how copyright works:
> Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code
This is simply not true. The reason why the "clean room" concept exists is precisely since actually the law recognizes that independent implementations ARE possibile. The "clean room" thing is a trick to make the litigation simpler, it is NOT required that you are not exposed to the original code. For instance, Linux was implemented even if Linus and other devs where well aware of Unix internals. The law really mandates this: does the new code copy something that was in the original one? The clean room trick makes it simpler to say, it is not possible, if there are similar things it is just by accident. But it is NOT a requirement.
Regardless of the legal interpretations, I think it's very worrying if an automated AI rewrite of GPLed code (or any code for that matter) could somehow be used to circumvent the original license. That kinda takes out the one stick the open source community has to force soulless multinationals to contribute back to the open source projects they use.
I’m genuinely surprised to see this not discussed more by the FOSS community. There are so many ways to blow past the GPL now:
1. File by file rewrite by AI (“change functions and vars a bit”)
2. One LLM writes a diff language (or pseudo code) version of each function that a diff LLM translates back into code and tests for input/output parity
The real danger is that this becomes increasingly undetectable in closed source code and can continue to sync with progress in the GPLed repo.
I don’t think any current license has a plausible defense against this sort of attack.
9 replies →
It's worrying, but it's consistent with how copyright law is currently written. Laws haven't caught up with what technology is currently capable of yet. The discussion should be whether, and if so how, our laws should be tweaked to stop this from getting out of hand, IMO.
If the AI is good enough to truly implement the whole thing to a similar level of reliability without copying it then who cares. At that point you should be able to decompile any program you want and find enough information inside that an AI can go write a similar quality program from the vague information about the call graph. We've transcended copyright in computer code.
If it can't and it costs a bunch of money to clean it up then same as always.
OTOH if what is actually happening is just that it is rewording the existing code so it looks different then it is still going to run afoul of copyright. You can't just rewrite harry potter with different words.
Note that even with Google vs oracle it was important they didn't need the actual code just the headers to get the function calls were enough. Yes it's true that the clean room isn't required but when you have an AI and you can show that it can't do it a second time without looking at the source (not just function declarations) that's pretty strong evidence.
1 reply →
Take AI out…if a person can do it, which they can, the situation hasn’t changed. Further, it was a person who did it, with the assistance of AI. Also, the concept that you “can’t be exposed to the code before writing a compatible alternative” is utterly false in their arguments. In fact, one could take every single interface definition they have defined to communicate and use those interfaces directly to write their own, because in fact this i(programmatic) interface code is not covered by copyright (with an implicit fair use exemption due to the face the software cannot operate without activating said interfaces). The Java lawsuit set that as precedent with JDK. A person could have absolutely rewritten this software using the interfaces and their knowledge, which is perfectly legal if they don’t literally copy and re-word code. Now, if it IS simply re-worded copies of the same code and otherwise the entire project structure is basically the same, it’s a different story. That doesn’t sound like what happened.
Finally, how exactly do people think corporations rewrite portions of code that were contributed before re-licensing under a private license? It is ABSOLUTELY possible to rewrite code and relicense it.
Edit: Further, so these people think you contribute to a project, that project is beholden to your contribution permanently and it can never be excised? That seems like it would blatantly violate their original persons rights to exercise their own control of the code without those contributions, which is exactly the purpose of a rewrite.
1 reply →
It cuts both ways. You can write a GPL version of a proprietary or permissively licensed program. The only difference is the effort of the rewrite is (theoretically) easier.
(I have my doubts the rewrite is a reasonably defect free replacement)
It’s less worry to me given that a year ago this would have been exceptionally harder to do, requiring a lot more time and effort and been more costly. A year from now it will be even easier. All of this means that one aspect of the mission that brought about the need for a license like this is now fundamentally easier whether or not the license is used. There can be less worry about software locked up in closed source overall.
If automated AI rewrites are generally feasible, then the marginal price of nearly all software trends to zero.
1 reply →
True, but if that is found to be how it works then an automated AI rewrite of closed-source code is just as unbound by the original license. Which is a much bigger win for the open-source community, since any closed-source software can become the inspiration for an open-source project.
Soulless multinationals often want to share costs with other soulless multinationals, just like individuals do. So I think there will always be publicly shared code. The real question is whether this code will be worth much if it can be implemented so quickly by a machine.
1 reply →
If it actually is a rewrite it's not "circumventing" it's just a new thing
That kinda takes out the one stick the open source community has to force soulless multinationals to contribute back to the open source projects they use.
I'll trade that stick for what GenAI can do for me, in a heartbeat.
The question, of course, is how this attitude -- even if perfectly rational at the moment -- will scale into the future. My guess is that pretty much all the original code that will ever need to be written has already been written, and will just need to be refactored, reshaped, and repurposed going forward. A robot's job, in other words. But that could turn out to be a mistaken guess.
8 replies →
Neither does the maintainer that claims a mechanical test of structural similarities can prove anything either waybwith regard to whether legally it is a derivative work (or even a mechnaical copy without the requisite new creative work to be a derivative work.)
And then Pilgrim is again wrong by saying that the use of Claude definitively makes it a derivative work because of the inability to prove it the work in question did not influence the neurons involved.
It is all dueling lay misreadings of copyright law, but it is also an area where the actual specific applicable law, on any level specific enough to cleanly apply, isn’t all that clear.
I think this is a bit too broad. There are actually three possible cases.
When there is similar code, the only defense possible to prove that you have not copied the original is to show that your process is a clean room re-implementation.
If the code is completely different, then clean room or not is indeed irrelevant. The only way the author can claim that you violated their copyright despite no apparent similarity is for them to have proof you followed some kind of mechanical process for generating the new code based on the old one, such as using an LLM with the old code as input prompt (TBD, completely unsettled: what if the old code is part of the training set, but was not part of the input?) - the burden of proof is on them to show that the dissimilarity is only apparent.
In realistic cases, you will have a mix of similar and dissimilar portions, and portions where the similarity is questionable. Each of these will need to be analyzed separately - and it's very likely that all the similar portions will need to be re-written again if you can't prove that they were not copied directly or from memory from the original, even if they represent a very small part of the work overall. Even if you wrote a 10k page book, if you copied one whole page verbatim from another book, you will be liable for that page, and the author may force you to take it out.
> When there is similar code, the only defense possible to prove that you have not copied the original is to show that your process is a clean room re-implementation.
Yes, but you do not have to prove that you haven’t copied the original; you have to prove you didn’t infringe copyright. For that there are other possible defenses, for example:
- fair use
- claiming the copied part doesn’t require creativity
- arguing that the copied code was written by AI (there’s jurisdiction that says AI-generated art can’t be copyrighted (https://www.theverge.com/2023/8/19/23838458/ai-generated-art...). It’s not impossible judges will make similar judgments for AI-generated programs)
5 replies →
The burden of proof is completely uncharted when it comes to LLMs. Burden of proof is assigned by court precedent, not the Copyright Act itself (in US law). Meaning, a court looking at a case like this could (should) see the use of an LLM trained on the copyrighted work as a distinguishing factor that shifts the burden to the defense. As a matter of public policy, it's not great if infringers can use the poor accountability properties of LLMs to hide from the consequences of illegally redistributing copyrighted works.
2 replies →
I'm with you here, but I see another problem.
The expected functionality of chardet (detect the unicode encoding) is kind of fixed - apart from edge cases and new additions to unicode, you'd expect the original and new implementations to largely pass the same tests, and have a lot of similar code such as for "does this start with a BOM".
The fact that the JPlag shows such a low %overlap for an implementation of "the same interface" is convincing evidence for me that it's not just plagiarised.
If you let an LLM merely rephrase the codebase, that's like letting it rephrase the Harry Potter novels. Which, I'm pretty sure, would still be considered a copy under copyright law, not an original work, despite not copying any text verbatim.
But what if it didn’t summarize Harry Potter? What if it analyzed Harry Potter and came back with a specification for how to write a compelling story about wizards? And then someone read that spec and wrote a different story about wizards that bears only the most superficial resemblance to Harry Potter in the sense that they’re both compelling stories about wizards?
This is legitimately a very weird case and I have no idea how a court would decide it.
1 reply →
Ok sure, in the alternative, here's the argument:
The AI was trained with the code, so the complete rewrite is tainted and not a clean room. I can't believe this would need spelling out.
"Tainted rewrite" isn't a legal concept either. You have to prove (on balance of probabilities - more likely than not) that the defendant made an unauthorized copy, made an unauthorized derivative work, etc. Clean-room rewriting is a defense strategy, because if the programmer never saw the original work, they couldn't possibly have made a derivative. But even without that, you still have to prove they did. It's not an offence to just not be able to prove you didn't break the law.
2 replies →
As other pointed out, the notion of "clean room" rewrites is to make a particularly strong case of non-infringement. It doesn't mean that anything other than a clean room implementation is an infringement.
This is interesting and I'm not sure what to make of it. Devil's advocate: the person operating the AI also was "trained with the code," is that materially different from them writing it by hand vs. assisted by an LLM? Honestly asking, I hadn't considered this angle before.
2 replies →
Reread the parent: clean room is not required.
1 reply →
I’m surprised they think the AI generated rewrite is even copyrightable.
Given that LLMs were trained on the repository directly, it's not just the case that anything made by the LLM is a derivative work, the LLM ITSELF is a derivative work. After all, they all are substantially based on GPL licensed works by others. The standard courts have always used for "substantially based" by the way, is the ability to extract from the new work anything bigger than an excerpt of the original work.
So convincing evidence, by historical standards, that ChatGPT, Gemini, Copilot AND Claude are all derivative works of the GPL linux kernel can be gotten simply by asking "give me struct sk_buff", then keep asking until you're out of the headers (say, ask how a network driver uses it).
That means if courts are honest (and they never are when it comes to GPL) OpenAI, Google and Anthropic would be forced to release ALL materials needed to duplicate their models "at cost". Given how LLMs work that would include all models, code, AND training data. After all, that is the contract these companies entered into when using the GPL licensed linux kernel.
But of course, to courts copyright applies to you when Microsoft demands it ($30000 per violation PLUS stopping the use of the offending file/torrent/software/... because such measures are apparently justified for downloading a $50 piece of software), it does not apply to big companies when the rules would destroy them.
The last time this was talked about someone pointed out that Microsoft "stole", as they call it, the software to do product keys. They were convicted for doing that, and the judge even increased damages because of Microsoft's behavior in the case.
But there is no way in hell you'll ever get justice from the courts in this. In fact courts have already decided that AI training is fair use on 2 conditions:
1) that the companies acquired the material itself without violating copyright. Of course it has already been proven that this is not the case for any of them (they scraped it without permission, which has been declared illegal again and again in the file sharing trials)
2) that the models refuse to reproduce copyrighted works. Now go to your favorite model and ask "Give me some code written by Linus Torvalds": not a peep about copyright violation.
... but it does not matter, and it won't matter. Courts are making excuses to allow LLM models to violate any copyright, the excuse does not work, does not convince rational people, but it just doesn't matter.
But of course, if you thought that just because they cheat against the law to make what they're already doing legal, they'll do the same for you, help you violate copyright, right? After all, that's how they work! Ok now go and ask:
"Make me an image of Mickey Mouse peeling a cheese banana under an angry moon"
And you'll get a reply "YOU EVIL COPYRIGHT VILLAIN". Despite, of course, Mickey Mouse no longer being covered under copyright!
And to really get angry, find your favorite indie artist, and ask to make something based on their work. Even "Make an MC Escher style painting of Sonic the Hedgehog" ... even that doesn't count as copyright violation, only the truly gigantic companies deserve copyright protection.
> Given that LLMs were trained on the repository directly, it's not just the case that anything made by the LLM is a derivative work, the LLM ITSELF is a derivative work.
That’s not how “derivative works”, well, work.
First of all, a thing can only be a derivative work if it is itself an original work of authorship.
Otherwise, it might be (or contain) a complete copy or a partial copy of one or more source works (which, if it doesn't fall into a copyright exception, would still be a at least a potential violation), but its not a derivative work.
4 replies →
The LLM would, under that argument, be a transformative derivative work, which has important fair use implications (that don’t exist in the chardet case)…
[dead]
This is correct. I think any author of a main chunk of code that they claim ownership to (which is probably all of us!) should at least study the basics of copyright law. Getting little details wrong can cost you time, money and eventually your business if you're not careful.
The argument that a rewrite is a copyright violation because they are familiar with the code base is not fully sound.
"Insider Knowledge" is not relevant for copyright law. That is more in the space of patent law then copyright law.
Or else a artist having seen a picture of a sunset over an empty ocean wouldn't be allowed to pain another sunset over an empty ocean as people could claim copyright violation.
Through what is a violation is, if you place the code side by side and try to circumvent copyright law by just rephrasing the exact same code.
This also means that if you give an AI access to a code base and tell it to produce a new code base doing the same (or similar) it will most likely be ruled as copyright violation as it's pretty much a side by side rewriting.
But you very much can rewrite a project under new license even if you have in depth knowledge. IFF you don't have the old project open/look at it while doing so. Rewrite it from scratch. And don't just rewrite the same code from memory, but instead write fully new code producing the same/similar outputs.
Through while doing so is not per-se illegal, it is legally very attackable. As you will have a hard time defending such a rewrite from copyright claims (except if it's internally so completely different that it stops any claims of "being a copy", e.g. you use complete different algorithms, architecture, etc. to produce the same results in a different way).
In the end while technically "legally hard to defend" != "illegal", for companies it's most times best to treat it the same.
> "Insider Knowledge" is not relevant for copyright law. That is more in the space of patent law then copyright law.
On the contrary. Except for discussions about punitive damages and so on, insider knowledge or lack thereof is completely irrelevant to patent law. If company A has a patent on something, they can assert said patent against company B regardless of whether any person in company B had ever seen or heard of company A and their patent. Company B could have a legal trail proving they invented their product that matches the patent from scratch with no outside knowledge, and that they had been doing this before company A had even filed their patent, and it wouldn't matter at all - company A, by virtue of filing and being granted a patent, has a legal monopoly on that invention.
In contrast, for copyright the right is intrinsically tied to the origin of a work. If you create a digital image that is entirely identical at the pixel level with a copyrighted work, and you can prove that you had never seen that original copyrighted work and you created your image completely independently, then you have not broken anyone's copyright and are free to sell copies of your own work. Even more, you have your own copyright over your own work, and can assert it over anyone that tries to copy your work without permission, despite an identical work existing and being owned by someone else.
Now, purely in principle this would remain true even if you had seen the other work. But in reality, it's impossible to convince any jury that you happened to produce, entirely out of your own creativity, an original work that is identical to a work you had seen before.
> But you very much can rewrite a project under new license even if you have in depth knowledge. IFF you don't have the old project open/look at it while doing so.
No, this is very much false. You will never be able to win a court case on this, as any significant similarity between your work and the original will be considered a copyright violation, per the preponderance of the evidence.
> In contrast, for copyright the right is intrinsically tied to the origin of a work. If you create a digital image that is entirely identical at the pixel level with a copyrighted work, and you can prove that you had never seen that original copyrighted work and you created your image completely independently, then you have not broken anyone's copyright and are free to sell copies of your own work.
This is not true. I will just give the example of the nighttime illumination of the Eiffel Tower:
> https://www.travelandleisure.com/photography/illegal-to-take...
> https://www.headout.com/blog/eiffel-tower-copyright/
4 replies →
If I read Mario Puzo’s The Godfather and then proceed to write a structurally identical novel with many of the same story beats and character types, it will not be difficult to convince a jury exposed to these facts that I’ve created a derivative work.
On the other hand, if I can prove to the jury’s satisfaction that I’ve never been exposed to Puzo’s work in any form, it’s independent creation.
To the contrary, there have been many cases of very similar novels with largely identical plot points and settings that survive copyright allegations, even if the author was exposed to the original work.
For a rather entertaining example (though raunchy, for a heads up): https://www.youtube.com/watch?v=zhWWcWtAUoY&themeRefresh=1
2 replies →
In the case of chardet though it wouldn't it be more like you were the publisher of the godfather novel, withdrawing it from print and releasing a novel with the same name with much of the same plot and characters but claiming the new version was an independent creation?
1 reply →
If the new maintainers used Claude as their “fancy code generator” (there’s a Claude.md file in the repository so it seems so) then it was almost certainly trained with the chardet source code.
> And don't just rewrite the same code from memory, but instead write fully new code producing the same/similar outputs.
How different does the new code have to be from the old code and how is that measured?
nobody can tell and this is how we entered this very turbulent modern times of "everything can be retold" without punishment. LLMs already doing it at large, while original author is correct in terms of the LGPL, it is nearly impossible to say how different should expression of an idea be to be considered separate one. this is truly fundamental philosophical question that may not have an easy answer.
This is a bad argument.
Think of a rewrite (by a human or an LLM) as a translation. If you wrote a book in English and somebody translated it into Spanish, it'd still be a copyright issue. Same thing with translations.
That's very different to taking the idea of a body of work. So you can't copyright the idea of a pirate taking a princess hostage and a hero rescuing her. That's too generic. But even here there are limits. There have been lawsuits over artistic works being too similar.
Back to software, you can't copyright the idea of photo-editing software but you can copyright the source code that produces that software. If you can somehow prompt an LLM to produce photo editing software or if a person writes it themselves then you have what's generally referred to as a "cleanroom" implmentation and that's copyright-free (although you may have patent issues, which is a whole separate issue).
But even if you prompted an LLM that way, how did the LLM learn what it needed? Was the source code of another project an input in its training? This is a legal grey area, currently. But I suspect it's going to be a problem.
Suchir Balaji, the OpenAI researcher who was found dead in his flat just before testifying against his employer, published an excellent article somehow related to this topic:
When does generative AI qualify for fair use?
https://suchir.net/fair_use.html
Balaji's argument is very strong and I feel we will see it tested in court as soon as LLM license-washing starts getting more popular.
Hate to be "that guy" but in a corrupt legal system, which ours is, none of this matters. Who has the influence and dollars to make the decision theirs is all that matters.
I think you could have an LLM produce a written English detailed description of the complete logic of the program and tests.
Then use another LLM to produce code from that spec.
This would be similar to the cleanroom technique.
Producing a copy of a copyrighted work through a purely mechanical process is clear violation of copyright. LLMs are absolutely not different from a copier machine in the eyes of the law.
Original works can only be produced by a human being, by definition in copyright law. Any artifact produced by an animal, a mechanical process, a machine, a natural phenomenon etc is either a derived work if it started from an original copyrighted work, or a public domain artifact not covered by copyright law if it didn't.
For example, an image created on a rock struck by lightning is not a copyright covered work. Similarly, an image generated by an diffusion model from a randomly generated sentence is not a copyrightable work. However, if you feed a novel as a prompt to an LLM and ask for a summary, the resulting summary is a derived work of said novel, and it falls under the copyright of the novel's owner - you are not allowed to distribute copies of the summary the LLM generated for you.
Whether the output of an LLM, or the LLM weights themselves, might be considered derived works of the training set of that LLM is a completely different discussion, and one that has not yet been settled in court.
Perhaps - but an argument might still be made that the result is a derivative work of the original, given that it's produced by feeding the original work through automated tooling.
But either way, deleting the original version from the repo and replacing it with the new version - as opposed to, say, archiving the old version and starting a new repo with the new version - would still be a dick move.
Assuming the second LLM hadn’t been trained on the existing codebase. Which in this case we can’t know, but can assume that it was.
Does the second LLM have the codebase in its training?
1 reply →
As part of my consulting, i've stumbled upon this issue in a commercial context. A SaaS company who has the mobile apps of their platform open source approached me with the following concern.
One of their engineers was able to recreate their platform by letting Claude Code reverse engineer their Apps and the Web-Frontend, creating an API-compatible backend that is functionally identical.
Took him a week after work. It's not as stable, the unit-tests need more work, the code has some unnecessary duplication, hosting isn't fully figured out, but the end-to-end test-harness is even more stable than their own.
"How do we protect ourselves against a competitor doing this?"
Noodling on this at the moment.
You're not describing anything new, you're describing progress. A company invests time and money and expertise into building a product, it becomes established, people copy in 1/10th of the time, the quality of products across the industry improve. Long before generative AI, Instagram famously copied Snapchat's stories concept in a weekend, and that is now a multi-multi-multi-billion contributor to Meta's bottom line.
As engineers, we often think only about code, but code has never been what makes a business succeed. If your client thinks that their businesses primary value is in the mobile app code they wrote, 1) why is it even open source? 2) the business is doomed.
Realistically, though, this is inconsequential, and any time spent worrying about this is wasted time. You don't protect yourself from your competitor by worrying about them copying your mobile app.
> You don't protect yourself from your competitor by worrying about them copying your mobile app.
They did not copy the mobile app. They copied the service.
1 reply →
You might be interested in the dark factory work here https://factory.strongdm.ai/
They do something very similar for some of their work. It’s hard to use external services so they replicate them and the cost of doing so has come down from “don’t be daft, we can’t reimplement slack and google drive this sprint just to make testing faster” to realistic. They run the sdks against the live services and their own implementations until they don’t see behaviour differences. Now they have a fast slack and drive and more (that do everything they need for their testing) accelerating other work. I’m dramatically shifting my concept of what’s expensive and not for development. What you’re describing could have been done by someone before, but the difficulty of building that backend has dropped enormously. Even if the application was closed you could probably either now or soon start to do the same thing starting with building back to core user stories and building the app as well.
You can view some of this as having things like the application as a very precise specification.
Really fascinating moment of change.
> It’s hard to use external services
I think it's interesting to add what they use it for and why its hard.
What they use it for:
- It's about automated testing against third party services.
- It's not about replicating the product for end users
Why using external services is hard/problematic
- Performance: They want to have super fast feedback cycles in the agentic loop: In-Memory tests. So they let the AI write full in-memory simulations of (for example) the slack api that are behaviorally equivalent for their use cases.
- Feasiblity: The sandboxes offered by these services usually have performance limits (= number of requests per month, etc) that would easily be exhausted if attached to a test harness that runs every other minute in an automated BDD loop.
> "How do we protect ourselves against a competitor doing this?"
If the platform is so trivial that it can be reverse engineered by an AI agent from a dumb frontend, what's there to protect against? One has to assume that their moat is not that part of the backend but something else entirely about how the service is being provided.
Interesting case, IANAL but sounds legal and legit. The AI did not have expose to the backend it re-implemented. The API itself is public and not protectable.
OTOH as of yesterday the output of the LLM isn't copyrightable, which makes licensing it difficult
24 replies →
No serious enterprise SaaS company differentiates themselves solely on the product (the products are usually terrible). It's the sales channel, the fact that you know how to bill a big company, the human engineer who is sent on site to deploy and integrate the product, the people on the support line 24/7, the regulatory framework that ensures the customer can operate legally and obtain insurance, the fact that there's a deep pool of potential hires who have used and understand the product. Those are the differentiators.
> "How do we protect ourselves against a competitor doing this?"
You can try patenting; but not after the fact. Copyright won't help you here. You can't copyright an algorithm or idea, just a specific form or implementation of it. And there is a lot of legal history about what is and isn't a derivative work here. Some companies try to forbid reverse engineering in their licensing. But of course that might be a bit hard to enforce, or prove. And it doesn't work for OSS stuff in any case.
Stuff like this has been common practice in the industry for decades. Most good software ideas get picked apart, copied and re-implemented. IBM's bios for the first PC quickly got reverse engineered and then other companies started making IBM compatible PCs. IBM never open sourced their bios and they probably did not intend for that to happen. But that didn't matter. Likewise there were several PC compatible DOS variants that each could (mostly) run the same applications. MS never open sourced DOS either. There are countless examples of people figuring out how stuff works and then creating independent implementations. All that is perfectly legal.
IBM never open sourced their BIOS, but they did publish complete source code listings:
https://bitsavers.org/pdf/ibm/pc/pc/6025008_PC_Technical_Ref...
https://bitsavers.org/pdf/ibm/pc/xt/1502237_PC_XT_Technical_...
https://bitsavers.org/pdf/ibm/pc/at/1502494_PC_AT_Technical_...
Between this and the fact that their PC-DOS (née MS-DOS) license was nonexclusive, I'm honestly not sure what they expected to happen.
The nature of early IBM PC advertising suggests to me that they expected the IBM name and established business relationships to carry as much weight as the specifications itself, and that "IBM PC compatible" systems would be no more attractive than existing personal computers running similar if not identical third-party software (PC-DOS wasn't the only example of IBM reselling third-party software under nonexclusive license), and would perhaps even lead to increased sales of first-party IBM PCs.
Which, in fact, they did, leading me to believe the actual result may have been not too far from their original intent, only with IBM capturing and holding a larger share of the pie.
If your backend is trivial enough to be implemented by a large language model, what value are you providing?
I know it's a provoking question but that answers why a competitor is not a competitor.
I suspect you're underestimating the capabilities of today's LLMs.
> "How do we protect ourselves against a competitor doing this?"
I have been thinking about this a lot lately, as someone launching a niche b2b SaaS. The unfortunate conclusion that I have come to is: have more capital than anyone for distribution.
Is there any other answer to this? I hope so, as we are not in the well-capitalized category, but we have friendly user traction. I think the only possible way to succeed is to quietly secure some big contracts.
I had been hoping to bootstrap, but how can we in this new "code is cheap" world? I know it's always been like this, but it is even worse now, isn't it?
Maybe a better question is:
How do our competitors protect themselves against us doing this?
Particularly if you're named "Google", "Amazon", "Microsoft", or "Apple".
I think the genie is out of the bottle on this one and there's really no putting it back.
There is a certain amount of brand loyalty and platform inertia that will keep people. Also, as you point out, just having the source code isn't enough. Running a platform is more than that. But that gap will narrow with time.
The broader issue here is that there are people in tech who don't realize that AI is coming for their jobs (and companies) too. I hope people in this position can maybe understand the overall societal issues for other people seeing their industries "disrupted" (ie destroyed) by AI.
The famous case Google vs Oracle may need to be re-evaluated in the light of Agents making API implementation trivial.
https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_....
"How do we protect ourselves against a competitor doing this?"
That's the neat thing: you don't!
> "How do we protect ourselves against a competitor doing this?"
DMCA. The EULA likely prohibits reverse engineering. If a competitor does that, hit'em with lawyers.
Or, if you want to be able to sleep at night, recognize this as an opportunity instead of a threat.
What about jurisdictions where reverse engineering is an inalienable right?
4 replies →
Nothing. This is why SaaS stocks took a dump last week.
Makes me wonder when AI will put the mobile phone OS duopoly to an end.
The copyright argument is a sidetrack both in the PR comment thread and here. The issue opened claims the new code is based on the old code, and therefore derivative, and therefore must be offered in a modified version of the source code under the previous license, LGPL. The complaint is the maintainers violated the terms of LGPL, that they must prove no derivation from the original code to legally claim this is a legal new version without the LGPL license. Claim is if they or Claude read the old code (or of course directly use any of it) it is a license violation. “… in the release 7.0.0, the maintainers claim to have the right to “relicense” the project. They have no such right; doing so is an explicit violation of the LGPL. Licensed code, when modified, must be released under the same LGPL license. Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code (i.e. this is not a "clean room" implementation).“ By this reasoning, I am genuinely asking (I’m not a license expert) if a valid clean room rewrite is possible, because at a minimum you would need a spec describing all behavior, which ses to require ample exposure to the original to be sufficiently precise.
That's not what a derivative work means, though. Being exposed to something doesn't mean you can't create original work which is similar to it (otherwise every song or artwork would be a derivative work of everything before it)
People do cleanroom implementations as a precaution against a lawsuit, but it's not a necessary element.
In fact, even if some parts are similar, it's still not a clear-cut case - the defendant can very well argue that the usage was 1. transformative 2. insubstantial to the entirety of work.
"The complaint is the maintainers violated the terms of LGPL, that they must prove no derivation from the original code to legally claim this is a legal new version without the LGPL license."
The burden of proof is on the accuser.
"I am genuinely asking (I’m not a license expert) if a valid clean room rewrite is possible, because at a minimum you would need a spec describing all behavior, which ses to require ample exposure to the original to be sufficiently precise."
Linux would be illegal if so (they had knowledge of Unix before), and many GNU tools are libre API-compatible reimplementations of previous Unix utilities :)
The copyright argument is the only relevant argument. If the new work is a derived work of the original, then it follows by definition that the new work is under the copryight of the original's author(s). Since the original chardet was distributed by its author(s) only under the LGPL, any copy/derivative of it that anyone else creates must be distributed only under the LGPL, per the terms of the LGPL.
Now, whether chardet 7.0.0 is a derivative of chardet or not is a matter of copyright law that the LGPL has no say on, and a rather murky ground with not that much case law to rely on behind it. If it's not, the new author is free to distribute chardet 7.0.0 under any license they want, since it is a new work under his copyright.
> Claim is if they or Claude read the old code (or of course directly use any of it) it is a license violation
The original code is part of claude's training material. With that intepretation of the LGPL AI is incapable of writing non LGPL derivatives. I like that interpretation.
I'm not certain I buy it, but I find it a little hard to motivate the training being fair use if used to regenerate the project in a different licence.
It's the same project/name, with a version bump.
Maintainers must not be able to change the license that original author chose, and based on which contributors made contributions. When one stepped up to be maintainer, it was a trustee role, not owner role.
It should be perfectly ok (by maintainer or anyone for that mater) to be inspired from a community project and build something from scratch hand-crafting/ AI sloping, as long as the imitation is given a new name/ identity.
What rubbed me off personally was maintainer saying "pin your dependncies to version 6.0.0 or 5.x.x", as if maintainer owns the project. maintainer role is more akin to serve the community, not rule.
If it is completely new, why not start a new project with new name? No one will object. And of course leave the old project behind to whoever is willing to maintain it. And if the new name project is better, people will follow.
A distinction should be made between ownership of the code and its copyright and ownership of the repository and associated distribution channels. As far as I know, there's no precedent to state that owning the former means ownership of the latter. The original author abandoned this project years ago and likely has no legal basis to maintain that the project itself stays LGPL, only that their code and derivatives of it stay LGPL. Unless it could be proven otherwise, a rewrite of the project under a different license made without directly referencing the original is likely well within the rights of the owners of the repository to do.
Say a community builds a hall with an explicit intent of community use only, led by single person or a group, and then a person/ a group is appointed as a caretaker. Caretaker of the community hall decides unilaterally to convert the hall into a business convention center, razing old building to ground and rebuilding, disregarding community wishes, to make hall business friendly. How would you react to this situation? Caretaker has the ownership of the community assets, including the ground on which the hall is standing?
my understanding of the situation is:
Is the caretaker paying from his own pocket to maintain the hall? no
Is the caretaker paying from his own pocket for community usage of the hall? no
Is the caretaker spending time to maintain the community hall? yes
Is caretaker obliged to spend time on community hall? no
Is caretaker free to stop spending time on community hall? yes.
Is caretaker free to raze current hall, build new hall on same ground for new purposes WITH community agreement? YES
Is caretaker free to raze current hall, build new hall on same ground for new purposes WITHOUT community agreement (even if paying all the bill)? NO
Is caretaker free to build another similar hall someplace else? YES
Reasoning of your comment is of someone who is hell bent on staking claims on community resources (like big companies) without having slightest concern of the wishes or well-being of the community. Not sure of the commenter's motive either, given the new account with just two comments, supporting such blatant disregard of basic human decency.
7 replies →
Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code.
Question: if they had built one using AI teams in both “rooms”, one writing a spec the other implementing, would that be fine? You’d need to verify spec doesn’t include source code, but that’s easy enough.
It seems to mostly follow the IBM-era precedent. However, since the model probably had the original code in its training data, maybe not? Maybe valid for closed source project but not open-source? Interesting question.
> Sounds like they didn’t build a proper clean room setup: the agent writing the code could see the original code.
It doesn't matter how they structure the agents. Since chardet is in the LLM training set, you can't claim any AI implementation thereof is clean room.
Yeah I mention that in the question.
Might still be valid for closed source projects (probably is).
I think courts would need to weigh in on the open source side. There’s legal precedent is that you can use a derived work to generate a new unique work (the spec derived for the copyrighted code is very much a derived work). There are rulings that LLMs are transformative works, not just copies of training data.
LLMs can’t reproduce their entire training set. But this thinking is also ripe for misuse. I could always train or fine-tune a model on the original work so that it can reproduce the original. We quickly get into statistical arguments here.
It’s a really interesting question.
2 replies →
So by that logic, you're not legally allowed to implement your own character detector and license it as your own if you've ever looked at chardet's source code? I'm confused. I thought copyright laws protect intellectual property as-is, not the impression it leaves on someone.
9 replies →
This seems right to me. If you ask a LLM to derive a spec that has no expressive element of the original code (a clean-room human team can carefully verify this), and then ask another instance of the LLM (with fresh context) to write out code from the spec, how is that different from a "clean room" rewrite? The agent that writes the new code only ever sees the spec, and by assumption (the assumption that's made in all clean room rewrites) the spec is purely factual with all copyrightable expression having been distilled out. But the "deriving the spec (and verifying that it's as clean as possible)" is crucial and cannot be skipped!
How would a team verify this for any current model? They would have to observe and control all training data. In practice, any currently available model that is good enough to perform this task likely fails the clean room criteria due to having a copy of the source code of the project it wants to rewrite. At that point it's basically an expensive lossy copy paste.
1 reply →
It requires the original project to not be in the training data for the model for it to be a clean room rewrite
6 replies →
Somewhat annoyingly, there's been research that suggests that models can pass information to each other via (effectively) steganographic techniques - specific but apparently harmless choices of tokens, wordings, and so on; see https://arxiv.org/abs/1712.02950 and https://alignment.anthropic.com/2025/subliminal-learning/ for some simple examples.
While it feels unlikely that a simple "write this spec from this code" + "write this code from this spec" loop would actually trigger this kind of hiding behaviour, an LLM trained to accurately reproduce code from such a loop definitely would be capable of hiding code details within the spec - and you can't reasonably prove that the frontier LLMs have not been trained to do so.
Yeah I think, the Compaq / IBM precedent can only superficially apply. It would be like having two teams only meet in a room full of documentation - but both teams crammed the source code the day before. (That, the source code you are "reverse engineering" is in the training data.) It doesn't make sense.
Also, it's weird that it's okay apparently to use pirated materials to teach an LLM, but maybe not to disseminate what the LLM then tells you.
Not if the codebase was included in training the implementer.
Answer: probably not, as API-topography is also a part of copyright
Edit: this is wrong
Didn't the Google - Oracle case about Java APIs in Android https://en.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_.... directly disprove this?
1 reply →
The courts decided that wasn’t true for IBM, Java and many other cases. API typography describes functionality, which isn’t copyrightable (IANAL).
Wasn't Oracle vs Google about all of that?
A comment from 2021:
> Unfortunately, because the code that chardet was originally based on was LGPL, we don't really have a way to relicense it. Believe me, if we could, I would. There was talk of chardet being added to the standard library, and that was deemed impossible because of being unable to change the license.
So the person that did the rewrite knew this was a dive into dangerous water. That's so disrespectful.
[flagged]
Why didn't he just start a new project?
> Licensed code, when modified, must be released under the same LGPL license. Their claim that it is a "complete rewrite" is irrelevant, since they had ample exposure to the originally licensed code (i.e. this is not a "clean room" implementation).
I don't think that the second sentence is a valid claim per se, it depends on what this "rewritten code" actually looks like (IANAL).
Edit: my understanding of "clean room implementation" is that it is a good defence to a copyright infrigement claim because there cannot be infringement if you don't know the original work. However it does not mean that NOT "clean room implementation" implies infrigement, it's just that it is potentially harder to defend against a claim if the original work was known.
I agree that (while the ethics of this are a different issue) the copyright question is not obviously clear-cut. Though IANAL.
As the LGPL says:
> A "work based on the Library" means either the Library or any derivative work under copyright law: that is to say, a work containing the Library or a portion of it, either verbatim or with modifications and/or translated straightforwardly into another language. (Hereinafter, translation is included without limitation in the term "modification".)
Is v7.0.0 a [derivative work](https://en.wikipedia.org/wiki/Derivative_work)? It seems to depend on the details of the source code (implementing the same API is not copyright infringement).
I was wondering how the existing case law of translated works, from one language to an other works here. It would at suggest that this is an infringement of the license especially because of the lack of creativity. But IANAL and of course no idea of applicable case law.
"Exposure" means here, I think, that they feed the 6.X code version to Claude.
the ai copy pasted the existing project. How can such a procedure not fall under copyright?
Especially now that ai can do this for any kind of intellectual property, like images, books or sourcecode. If judges would allow an ai rewrite to count as an original creation, copyright as we know it completely ends world wide.
Instead whats more likely is that no one is gonna buy that shit
>the ai copy pasted the existing project.
The change log says the implementation is completely different, not a copy paste. Is that wrong?
>Internal architecture is completely different (probers replaced by pipeline stages). Only the public API is preserved.
4 replies →
I came here to say this. While I agree with Mark that what they’re doing is not nice, I’m not sure it’s wrong. A clean-room implementation is one way the industry worked around licensing in the past (and present, I guess), but it’s not a requirement in law as far as I know.
I’m not sure that “a total rewrite” wouldn’t, in fact, pass muster - depending on how much of a rewrite it was of course. The ‘clean room’ approach was just invented as a plausible-sounding story to head off gratuitous lawsuits. This doesn’t look as defensible against the threat of a lawsuit, but it doesn’t mean it wouldn’t win that lawsuit (I’m not saying it would, I haven’t read or compared the code vs its original). Google copied the entire API of the Java language, and got away with it when Oracle sued. Things in a courtroom can often go in surprising ways…
[edit: negative votes, huh, that’s a first for a while… looks like Reddit/Slashdot-style “downvote if you don’t like what is being said” is alive and well on HN]
I spent like two minutes looking at the diff between the original and the supposed "clean room" implementation [1] and already found identical classes, variable names, methods, and parameters. It looks like there was no actual attempt at clean-rooming this, regardless of whether that "counts".
[1]https://github.com/chardet/chardet/compare/6.0.0.post1...7.0...
Lol at the statement that "clean room" would have been invented to scare people from suing. It's the opposite: clean room is a fairly-desperate attempt to pre-empt accusations in court when it is expected that the "derivative" argument will be very strong, in order to then piggyback on the doctrine about interoperability. Sometimes it works, but it's a very high bar to clear.
I thought we were debating if it was legal, not if it's wrong. The law is about creativity. Was this creative or a more mechanical translation?
It will hold up in court. The line of argument of “well I went into a dark room with only the first Harry Potter book and a type writer and reproduced the entire work, so now I own the rewrite” doesn’t hold up in court, it doesn’t either when when you put AI in the mix. It doesn’t matter if the result is slightly different, a judge will rule based on the fact that this even is literally what the law is intended to prevent, it’s not a case of which incantation or secret sentence you should utter to free the work of its existing license.
> “well I went into a dark room with only the first Harry Potter book and a type writer and reproduced the entire work, so now I own the rewrite”
This is not a good analogy.
A "rewrite" in context here is not a reproduction of the original work but a different work that is functionally equivalent, or at least that is the claim.
3 replies →
“Mr Teacher, how many words do I have to change after copy pasting wikipedia so its not plagiarism?” has grown up and entered the workforce.
Pin your dependency versions people! With hashes at this point, cant trust anybody out here.
There's a subtext in your point that I want to expand on.
Tech people, particularly engineers, tend to make a fundamental error when dealing with the law that almost always causes them to make wrong conclusions. And that error is that they look for technical compliance when so much of the law is subjective and holistic.
An example I like to use is people who do something illegal on the Internet and then use the argument "you can't prove I did it (with absolute certainty)". It could've been someone who hacked your Wifi. You don't know who on the Wifi did it, etc. But the law will look at the totality of the evidence. Did the activity occur when you were at home and stop when you weren't? How likely are alternative explanations? Etc.
All of that will be considered based on some legal standard depending on the venue. In civil court that tends to be "the preponderance of the evidence" (meaning more likely than not) while in criminal court it's "beyond a reasonable doubt" (which is a much higher standard).
So, using your example, an engineer will often fall into a trap of thinking they can substitute enough words to have a new original work, Ship of Theseus-like. And the law simply doesn't work that way.
So, when this gets to a court (which it will, it's not a question of "if"), the court will consider how necessary the source work was to what you did. If you used it for a direct translation (eg from C++ to Go) then you're going to lose. My prediction is that even using it in training data will be cause for a copyright claim.
If you use Moby Dick in your training data and ask an LLM to write a book like Moby Dick (either explicitly or implicitly) then you're going to have an issue. Even if you split responsibilities so one LLM (training on Moby Dick) comes up with a structure/prompt and another LLM (not trained on Moby Dick) writes it, I don't think that'll really help you avoid the issue.
> So, when this gets to a court (which it will, it's not a question of "if"), the court will consider how necessary the source work was to what you did. If you used it for a direct translation (eg from C++ to Go) then you're going to lose. My prediction is that even using it in training data will be cause for a copyright claim.
This has a lot of similarity to when colorization of film started popping up. Did colorizing black and white movies suddenly change the copyright of the film? At this point is seems mostly the courts say no. But you may find sometimes people rule the other way and say yes. But it takes time and a lot of effort to get what in general people want.
But basically if you start with a 'spec' then make something you probably can get a wholly owned new thing. But if you start with the old thing and just transform it in some way. You can do that. But the original copyright holders still have rights too to the thing you mangled too.
If I remember right they called it 'color of copyright' or something like that.
The LLM bits you are probably right. But that has not been worked out by the law or the courts yet. So the courts may make up new case law around it. Or the lawmakers might get ahead of it and say something (unlikely).
2 replies →
> And that error is that they look for technical compliance when so much of the law is subjective and holistic.
I know it sounds like an oversimplification, but "got off on a technicality" is a common thing among the well-connected and well-heeled. Sure, us nerds probably focus too much on the "technicality" part, since we are by definition technical, but the rest is wishy-washy, unfair BS as far as many of our brains work much of the time.
1 reply →
Everyone armchair debating the licensing side of this frustrates discussion, because the only licensing discussion that matters is the one in front of a court. Until and unless one happens, this is just a boring hobby.
Otherwise all this rewrite accomplishes is a 2.3% accuracy improvement and some performance gains that might not be relevant in production, in exchange for a broken test suite, breaking changes, and unnecessary legal and ethical risks pushed out as an update to what was already a stable project.
If it's truly a sufficiently separate project that it can be relicensed from LGPL, then it could've just been _a fully separate project with a new identity_, and the license change would've been at least harder to challenge. Instead, we're here.
Wow that's hot. I was not aware that you need to be "untainted" by the original LGPL code. This could mean that...
All AI generated code is tainted with GPL/LGPL because the LLMs might have been taught with it
Being completely untainted is the standard many reimplementations set for themselves to completely rule out legal trouble. For example ReactOS won't let you contribute if you have ever seen Windows code. Because if you have never seen it, there can be no allegation that you copied it.
That is however stricter than what's actually legally necessary. It's just that the actual legal standard would require a court ruling to determine if you passed it, and everyone wants to avoid that. As a consequence there also aren't a lot of court cases to draw similarities to
> For example ReactOS won't let you contribute if you have ever seen Windows code. Because if you have never seen it, there can be no allegation that you copied it.
I've heard this called in some circles "The curse of knowledge." The same thing applies to emulator developers, especially N64 developers (and now Nintendo emulator developers in general) after the Oman Archive and later Gigaleaks. There's an informal "If you read this, you can NEVER directly contribute to the development of that emulator, ever."
This comes to a head when a relatively unknown developer starts contributing oddly specific patches to an emulator.
"Taint" requires that the code is demonstratably derivative from the *GPL licensed work.
This is actually harder standard than some people think.
The absolute clean room approaches in USA are there because they help short circuit a long lawsuit where a bigger corp can drag forever until you're broken.
It's harder than some people think, but the author does a lot of the work when he names the resulting artifact "chardet v7.0.0". If I thought I was writing the kind of arms-length reimplementation that's required, I would never put it into the versioning scheme of the original, come on.
1 reply →
[flagged]
Does "lonely" in this case encompass people who've formed relationshios with said LLMs?
I'm not lonely! And I stopped shouting that since 24, because you know :/
Not a lawyer, but that always seemed naively correct to me.
However, the copyright system has always be a sham to protect US capital interests. So I would be very surprised if this is actually ruled/enforced. And in any case american legislators can just change the law.
On a side note, it is interesting to see Mark Pilgrim rise from the "dead": https://en.wikipedia.org/wiki/Mark_Pilgrim#%22Disappearance%...
His Python books, although a bit dated, are something I still recommend to new Python programmers.
Let's see how much time it takes for the Wikipedia page to mention the "a2mark" GitHub user :)
(I can hear a "challenge accepted" from some random HNer already)
I torn on where the line should be drawn.
If the code is different but API compatible, Google Java vs Oracle Java case shows that if the implementation is different enough, it can be considered a new implementation. Clean room or not.
That whole clean room argument makes no sense. Project changed governance and was significantly refactored or reimplemented... I think the maintainers deserve to call it their own. Original-pre MIT release can stay LGPL.
I don't think this is a precedent either, plenty of projects changed licenses lol.
I keep kind mixing them up but the GPL licenses keep popping up as occasionally horror stories. Maybe the license is just poorly written for today's standards?
> plenty of projects changed licenses lol.
They usually did that with approval from existing license holders (except when they didn't, those were the bad cases for sure).
No. Because they couldnt have done any of that refactoring without a licence to do so, and that licence forbids them from relicencing it.
14 replies →
Governance change or refactoring don’t give you a right to relicense someone else’s work. It needs to be a whole new work, which you own the copyright to.
1 reply →
No, that defeats the entire purpose of GPL licenses
It's not clear at all why the current maintainers wanted/needed this re-licensing. I guess that their employee, Monarch Money, wants to use derivative work in their application without releasing the changes? It was already LGPL, perfect for a library, not GPL.
Python wouldn't take LGPL code in the standard library. And Dan Blanchard imagined more people would want to work on it.[1]
[1] https://github.com/chardet/chardet/issues/327#issuecomment-4...
That take and the link within it to https://github.com/chardet/chardet/issues/36 which is from 2014 (!!) and revived in 2021 (!) make me think now that Dan Blanchard is in good faith, but still acted in a naive manner. Rewriting from "scratch" with Claude Code and basically taking over the project by changing the license, all in one single pull request, well... it's not going to end well on the public relations side.
"I prefer MIT/BSD licenses just because they're simpler"[0]
Seems like there is no real point, just vibes.
[0] https://github.com/chardet/chardet/issues/36
Copyright protects "authorship", not functionality. Patent protects functional elements.
A rewrite based on functional equivalency is not infringing on the copyright as long as no creative expression was copied. That was the heart of the Google case, whether the API itself was creative expression or functionality.
There are many aspects to what can be considered creative expression, including names, organization, non-functional aspects. An algorithm would not be protected expression. If an AI can write it without reference to the original source code, using only documented behavior, then it would not be infringing (proving that it didn't copy anything from training data might be tough though). It also would not itself be copyrightable, except for elements that could be traced back as "authorship" to the humans who worked with the AI.
If LLMs can create GOOD software based only on functionality, not by copying expression, then they could reproduce every piece of GPL software and release it as Public Domain (which it would have to be if no human has any authorship in it). By the same principle that the GPL software wasn't infringing on the programs they copied functionality from, neither would the AI software. That's a big IF at this point, though, the part about producing GOOD software without copying.
Look, forget the details, step back and consider the implications of the principle.
Someone should not be able to write a semi-common core utility, provide it as a public good, abandon it for over a decade, and yet continue to hold the rest of the world hostage just because of provenance. That’s a trap and it’s not in any public interest.
The true value of these things only comes from use. The extreme positions for ideals might be nice at times, but for example we still don’t have public access to printer firmware. Most of this ideology has failed in key originating goals and continues to cause headaches.
If we’re going to share, share. If you don’t want to share, don’t. But let’s not setup terminal traps, no one benefits from that.
If we flip this back around though, shouldn’t this all be MPL and Netscape communications? (Edit: turns out they had an argument about that in the past on their own issue tracker: https://github.com/chardet/chardet/issues/36)
LGPL means "gift to the world". The license ensures that any modification/improvements stay a gift to the world.
People not being okay with having to share their improvements not being able to use the software is by design.
I don't get how you get from there to some sinister hostage taking situation.
Also everyone that contributes to the previous LGPL verison probably contributed under LGPL only, so it is now just one guy...
LGPL applies to the LGPL’d code, not to every piece of code someone might add to the repository or under the same name implicitly.
The claim being made is that because some prior implementation was licensed one way, all other implementations must also be licensed as such.
AIUI the code has provenance in Netscape, prior to the chardet library, and the Netscape code has provenance in academic literature.
Now the question of what constitutes a rewrite is complex, and maybe somewhat more complex with the AI involvement, but if we take the current maintainers story as honest they almost certainly passed the bar of independence for the code.
Fork it?
That certainly probably would have reduced the noise.
> chardet 7.0 is a ground-up, MIT-licensed rewrite of chardet. Same package name, same public API
Licensing aside, morally you don't rewrite someone else's project with the same package name.
I've got some serious moral questions about "rewrite your own widely used project from scratch with the same package name"... but I don't think it's fair to call this "someone else's project" when the OP has apparently been the only maintainer working on the project for 13 years...
Creator is still the creator and retains rights to the name.
Isn't the real issue here that tons of projects that depend on the "chardet" now drag in some crappy still unverified AI slop? AI forgery poisoning, IMHO.
Why does this new project here needed to replace the original like that in this dishonourable way? The proper way would have been to create a proper new project.
Note: even Python's own pip drags this in as dependency it seems (hopefully they'll stick to a proper version)
This indeed the real issue (not the AI angle per se, but the wholesale replacement. The licensing issue is real, but less important IMO).
Half a million lines of code have been deleted and replaced over the course of four days, directly to the main branch with no opportunity for community review and testing. (I've no idea whether depending projects use main or the stable branch, but stable is nearly 4 years old at this point, so while I hope it's the version depending projects use, I wouldn't put money on it.)
The whole thing smells a lot like a supply chain attack - and even if it's in good faith, that's one hell of a lot of code to be reviewed in order to make sure.
Woah. As someone not in this particular community but dependent on these tools this is exactly the terrifying underbelly we've all discussed with the user architecture of tools like pip and npm. It's horrifying that a major component just got torn apart, rebuilt, and deployed to anyone who uses those python ecosystems (... many millions? ... billions of people?)
The test coverage is going to be entirely different, unless of course they copied the tests, which would then preclude them from changing the license. They didn't even bother to make sure the CI passed on merging a major version release https://github.com/chardet/chardet/actions/runs/22563903687/...
The drop"-in" compatibility claims are also just wrong? I ran it on the old test suite from 6.0 (which is completely absent now), and quickly checking:
- the outputs, even if correctly deduced, are often incompatible: "utf-16be" turns into "utf-16-be", "UTF-16" turns into "utf-16-le" etc. FWIW, the old version appears to have been a bit of a mess (having had "UTF-16", "utf-16be" and "utf-16le" among its outputs) but I still wouldn't call the new version _compatible_,
- similarly, all `ascii` turn into `Windows-1252`
- sometimes it really does appear more accurate,
- but sometimes it appears to flip between wider families of closely related encodings, like one SHIFT_JIS test (confidence 0.99) turns into cp932 (confidence 0.34), or the whole family of tests that were determined as gb18030 (chinese) are now sometimes determined as gb2312 (the older subset of gb18030), and one even as cp1006, which AFAIK is just wrong.
As for performance claims, they appear not entirely false - analyzing all files took 20s, versus 150s with v6.0. However, looks like the library sometimes takes 2s to lazy initialize something, which means that if one uses `chardetect` CLI instead of Python API, you'll pay this cost each time and get several times slower instead.
Oh, and this "Negligible import memory (96 B)" is just silly and obviously wrong.
Yeah, there's really low quality code added if you take a look.
Huh, 7e25bf4 was a big commit.
https://github.com/chardet/chardet/commit/7e25bf40bb4ae68848...
Impossible to code inspect and verify. Someone else commented 'smells like a supply chain attack' and while I'm sure it's not intended to be, there is no way to verify. And who believes, in half a million lines of code, no bugs have been introduced?
Perhaps notable: years ago the original original chardet was rewritten with a different license: https://github.com/hsivonen/chardetng
AFAIK this was not a clean room reimplementation. But since it was rewritten by hand, into a different language, with not just a different internal design but a different API, I could easily buy that chardetng doesn't infringe while Python chardet 7 does.
You have to look from two sides this Moral: What is right or wrong? If they wanted to change the license, they could have made another project with another name, and nobody would care, but they wanted the reputation of the project.
Legal: How much are you willing to spend on litigation? The only real "protection" by copyright is in court.
Further, while the copyright of the original code and its derivatives are still owned by the original author does that hold true to the rights to the name and package namespace? Supposing this were indisputably a clean room implementation instead of an unclear one, would the maintainers then have a right to relicense under the same name? I would imagine that yes, they do have the right to relicense in that case because the copyright only applies to the code, not the project itself.
Other questions that haven't really been explored before also are maintained: the original author hasn't been involved in some time, technically the copyright of all code since still belongs to those authors who might be bound by LGPL but are also the only ones with the right to enforce it and could simply choose not to. What then?
I wonder if LLMs will push the industry towards protecting their IP with patents like the other branches of engineering rather than copyright. If you patent a general idea of how your software works then no rewrite will be able to lift this protection.
General patents aren't allowed.
If there wasn't a specific limitation against software patents, you could patent the process of "first we look at the first 3 bytes for a BOM, then we take statistics of the most frequently used bytes and match them against this table..." which is the software equivalent of "the device contains a large box full of water, underneath which is a flame and on the upper side is a pipe leading to a valve box which contains..."
Either one would still have to meet the requirements like being sufficiently non-obvious. The first steam engine was patented, even though you couldn't patent one any more.
Clean room implementations are not necessary to avoid copyright infringement.
Does anyone understand the intent behind the changed license on the package, why are the current maintainers trying to do it in the first place? What's actually going on?
Unicode detection is the kind of utility the language maintainers want in their package collection if not in the standard library, and programmers who have to do anything with "plain text" files might want to rely on.
Releasing a core library like this under a genuinely free licence (MIT) is a service to anyone working in the ecosystem.
Maybe. Enabling more GPL software to become proprietary isn't exactly a service.
3 replies →
Nobody that is not already writing (L)GPL licensed software wants to pull LGPL licensed libraries into his software. I'm not a lawyer. If you have it in separate object artefacts that you can dynamically link to, you should be fine. Everything else may be more difficult.
See what FFmpeg writes on this topic: https://ffmpeg.org/legal.html
https://news.ycombinator.com/item?id=47260749
With continuing the same repo, name, and reputation built on LGPL code.. you can’t keep the goodwill and drop the contract that created it :-)
I have successfully reproduced a few projects with LLM assistance via strict cleanroom rules and only working off public specifications.
Once you use a LLM, the room is no longer clean.
Does that hold up if it’s in an entirely different language?
Because I don’t think so
[flagged]
2 replies →
Does using the old version’s tests to create a new version make it a derivative work? That’s certainly some pretty tight coupling.
My gut feeling is no, a work is not a derivative of a compliance test suite that it passes and was used to guide it. But I'm not a lawyer.
We’re going to be in a very weird place:
* LLMs make it trivial to recreate almost any software using its test suite (maybe not a derivative work)
* LLM generated code has no copyright (according to current court interpretations)
Soon we will be able to make an unlicensed copy of anything if we have its test suite and a little money for tokens.
2 replies →
Interestingly the original author ported the code from the Mozilla C++ version which is licensed as "MPL 1.1 or LGPL" (it's a bit unclear as the readme says that but the license file mentions only MPL). So the author did already relicence the project in a way by licensing the port as LGPL only.
> licensing the port as LGPL only
They ported the LGPL version. There's no obligation to port any other, unless "MPL 1.1 or LGPL" is itself some kind of singular licence.
I am confused. In the USA, there has been a clear rule that machine-generated code cannot be copyright. If the "new implementation" was in fact created by Claude (which is my impression), then nobody holds any copyright to the code, and it cannot be licensed under any license at all.
I am sure I am missing something ... what is it?
This is effectively a contract. You can put anything you want in a contract, but contracts are enforceable to only to the extent they comply with the law (statutes, case law, the constitution, etc.)
So to settle this, someone needs to violate this license and get sued. Or maybe proactively sue?
Remember: this applies to all LLM-generated code, not just chardet. No LLM-generated code is copyrightable, and thus cannot be licensed. The legal challenge could come in any context where LLMs have been used and the code placed under any license (proprietary or otherwise).
Which is going cause a collision between the "not copyrightable" and "derived from copyrighted work" angles.
The machine cannot get authorship, but just as images created by humans with PhotoShop and all kinds of machinery are still copyrighted to the human creator - unless they explicitly set up some circumstances where the process of creation happens completely without them - code/software produced by a machine instructed by a human should get copyright (either original or derivative).
Unless the human is so far removed from the output. (And how far is far enough is probably very much depends on the circumstances and unless/until case law or Congress gives us some unifying criteria, it's going to be up to how the judge and the jury feels.)
..
For example someone set up a system where their dog ends up promoting some AI to make video games. This might be the closest to the case of that photo.
Though there the court ruled only that PETA (as friend of the monkey) cannot sue the photographer, because the monkey cannot be a copyright holder, but very importantly it didn't rule on the authorship of the photographer. (And thus the Wikimedia metadata which states that the image is in thr public domain, is simply their opinion.)
FastAPI's underlying library, Starlette, has been going through licensing shenanigans too lately: https://github.com/Kludex/starlette/issues/3042
Be really careful who you give your projects keys to, folks!
That doesn't seem related at all, this is just adding attribution, not changing the license through LLM-washing
Setting aside the legal questions, what a nasty thing to do. I would expect to see this kind of move from some big corpo, not an OSS maintainer. This feels like it runs counter to the whole open source ethos and undermines the idea that authorship means anything anymore.
If you license a project MIT, you're telling the legal system that you don't want to own it.
AI "translation" kills any incentive to open source code.
What's worse - disassembler+AI is good enough to "translate" the binary into working source code, probably in a different programming language than then original.
The README has clearly been touched by an LLM. Count the idiosyncrasies:
“chardet 7.0 is a ground-up, MIT-licensed rewrite of chardet. Same package name, same public API — drop-in replacement for chardet 5.x/6.x”
Do people not write anymore?
I finally had to mute r/isthisai on Reddit because there’s now a subset of people who see the hand of AI in everything. Could that be generated by a clanker? Sure, but it’s also exactly what I would write if I wanted a quick pitch for a library that addresses some immediate concerns. It’s also what I would focus on if the fact we had just finished a rebuild from scratch.
As Freud famously said, sometimes an em dash is just an em dash.
FWIW, I don't think there's even a room for interpretation here, given the commit that created the README (and almost all commits since the rewrite started 4 days ago) is authored by
> dan-blanchard and claude committed 4 days ago
1 reply →
We are both trying to steer an internet towards a place we want to be.
The em dash is just a bonus, the grammatical structure is the giveaway. I'd invite Blanchard to argue that it wasn't LLM-generated.
I use AI tooling all day every day and can easily pick out when something was written by most popular modern models. I welcome an agentic web; it's the inevitable future. But not like this. I want things to get better, not worse.
For me, some projects I start by writing a readme.txt by hand. That saves me time in cases I realize I'd be making something pointless. (I don't use chatbots when coding though)
The law is what a court says it is; there is precedent for decisions on human rewrites but LLM (assisted) code might still be fairly uncharted territory.
This is not unprecedented, TCC relicensed part of its code by being approved by all authors:
https://repo.or.cz/tinycc.git/blob/3d963aebcd533da278f086a3e...
The interesting part is that the original author is against it but some people claims it could be a rewrite and not a derivative work.
I don't know the legal basis of everything but it's definitly not morally correct toward the original author.
If the code is written by an Ai they can't copyright it. It is all public domain.
surely this can also be used to turn proprietary software into free
If you can afford to defend yourself in court
maybe we are now heading to the new world where software become 'free goods' like air?
maybe we are heading to a new world where software become "free goods" like air?
See also,
Can coding agents relicense open source through a “clean room” implementation of code?
https://news.ycombinator.com/item?id=47264043
I think Mark Pilgrim misrepresents the legal situation somewhat: The AI rewrite does not legally need to be a clean room implementation (whatever exactly that would even mean here).
That is just the easiest way to disambiguate the legal situation (i.e. the most reliable approach to prevent it from being considered a derivative work by a court).
I'm curious how this is gonna go.
Another tangent that I didn't see in the thread is that the Supreme Court just confirmed a ruling that LLM created art isn't copyrightable since the author must be human for copyright to apply.
If the new code was generated entirely by an LLM, can it be licensed at all? Or is it automatically in the public domain?
The US supreme court declined to hear a challenge to the claim that if you explicitly disclaim any human input to make a point then the art isn't copyrightable in the US. The copyrightability of "actually I had some design input" is still up in the air in the US, and copyrightability in general is still up in the air in probably the entire rest of the world as well as every court in the US outside of the DC circuit (because the supreme court declining to hear a case does not constitute an endorsement of the lower courts ruling or create precedent).
There's absolutely nothing stopping you granting a license to public domain work... granting a license is just waiving rights that the author might have to sue for copyright infringement under certain circumstances...
Personally I'd be unwilling to use this work without the license, because I would not be confident that it was public domain.
It would be in the public domain. Wouldn't really matter all that much if the end goal was to get it included in the Python standard library, but the whole "Copyright (c) 2024 Dan Blanchard" in the license file would just be BS.
The big question is whether or not is it a derivative work of an LGPL project. If it is, then it's just an outright copyright violation.
Another day in the post-copyright world. Surely someone somewhere is already using this to test the effect of copyright laws, should we decide to go back to that world.
[dead]
Licenses are cancer and the enemy of opensource.
I think it's just the GPL family of licenses that tend tend to cause most problems. I appreciate their intent, but the outcome often leaves a lot to be desired.
The GPL exists for the benefit of end users, not developers. It being a chore for developers who want to deny their users the software freedoms is a feature, not a bug.
9 replies →
If you have ill intentions or maybe you're a corporation that wants to use someone else's work for free without contributing anything back, then yes, I can see how GPL licenses "tend to cause problems".
Why? What's your problem with them? They do exactly what they're supposed to do, to ensure that future derivatives of the source code have to be distributed under the same license and distribution respects fundamental freedoms.
If the GPL causes you problems, then it's working as intended.
I like to think about GPL as a kind of an artistic performance and an elaborate critique of the whole concept of copyright.
Like, "we don't like copyright, but since you insist on enforcing it and we can't do anything against it, we will invent a clever way to use your own rules against you".
3 replies →
Wtf are these comments? A LGPL licensed project, guaranteed to be free and open source, being LLM-washed to a permissive license, and GPL is the problem here?
They are literally stealing from open source, but it's the original license that is the issue?
3 replies →
And what exactly are some of these problems?
There would be no open source without the gpl
The GPL was historically of course a counter-reaction to copyrightability of code. Hence "copyleft" etc.
MIT and BSD are doing just fine.
The 25519 crypto package that's built into practically everything these days (SSH, TLS, most e2e messaging) was released as both a spec and a C reference implementation _in the public domain_.
Nothing lasts forever. We now have tools that let us fulfill the ultimate goal of open source.
GPL was a means, not an end.
1 reply →
Open source as a concept is intertwined with the concept of a license.
Interesting.
While I am obviously Team GPL and not team "I 'rewrote' this with AI so now it's mine", I'm team anti-fork, and definitely not team 'Chardet'.
Forking should be a last resort, one better option is to yeet the thing entirely.
And chardet lends itself perfectly for this, using chardet is a sign of an issue and low craftmanship, either by the developer using chardet, or the developer that failed to signal the encoding of their text. (See Joel Spolsky's "The absolute minimum every developer should know about character encoding") (And let's be honest, it's probably the developers problem, not everything is someone else's fault.)
Just uninstall this thing where you can, and avoid installing it always, because you always can.
You know I'm right. I will not be replying to copium
The even more concerning news to me is that chardet 7.0 is now vibe coded AI slop, as documented in the PR for the rewrite [1]:
> I put this together using Claude Code with Opus 4.6 with the amazing https://news.ycombinator.com/item?id=47259177
I feel like the author is missing a huge point here by fighting this. The entire reason why GPL and any other copyleft license exists in the first place is to ensure that the rights of a user to modify, etc a work cannot be ever taken away. Before, relicensing as MIT - or any other fully permissive license - would've meant open doors to apply restrictions going forward, but with AI this is now a non-issue. Code is now very cheap. So the way I see this, anyone who is for copyleft should be embracing AI-created things as not being copyrightable (or a rewrite being relicensable) hard*.
The user is the end-user of the product. If the relicensing means that someone down the line receives a close-down binary application that he cannot modify, that's a violation of the user's rights.
But it's a non-issue as said user can just have AI reverse engineer said binary. Or reimplement something with the same specs. That's what it means for code to be cheap.
2 replies →
That is still true, but it was more relevant back when "user" meant "programmer at another university". The "end-user" for most software is not a programmer these days.
Depends on who wants to take what away.
If I release blub 1.0.0 under GPL, you cannot fork it and add features and release that closed-source, but I can certainly do that as I have ownership. I can't stop others continuing to use 1.0.0 and develop it further under the GPL, but what happens to my own 1.1.0 onwards is up to me. I can even sell the rights to use it closed-source.
You can do whatever you want. And someone else can - and will, if it's worth it - now do a deep analysis and have it reimplemented with said analysis, all via LLM. And particularly, since the law is now that AI creations can't be copyrighted, that reimplementation will be firmly in public domain. Nobody will have to even look at 1.1.0. See Valkey as one of the more recent examples, and there isn't even any LLM involved there.
Well put. It seems like a lot of people have tunnel vision on this. It's going to take a while for people to realize that copyright is obsolete, especially in its current form.
Code is only cheap with AI because AI ignores the law.
Laws change, and it's also law that now says AI-generated works can't be copyrighted, which makes everything even cheaper.
> 12-stage detection pipeline
What is this recent (clanker-fueled?) obsession to give everything fancy computer-y names with high numbers?
It's not a '12 stage pipeline', it's just an algorithm.
Isn’t it? I mean 12 stage pipeline has a very specific meaning to me in this area, and is not a new way of describing something. The release notes description sounds like a multi stage pipeline.
Do you know this kind of area and are commenting on the code?
"ok chatgpt, what name do i give to this algorithm, so it sounds fancy and advanced?"