Comment by alightsoul
17 hours ago
The only way to have agi or any ai is to throw away all ethics then. So why are ai companies shredding books instead of using Anna's archive? Depending on your stance, one is more ethical than the other, but not both. And not neither of them.
> why are ai companies shredding books instead of using Anna's archive?
I'm pretty sure they've already consumed everything on Anna's Archive, The Pirate Bay etc., and they are now buying books to fill the gaps and to legally launder the corpus they already had. It has little to do with ethics (one could also question the ethics of buying rare, second-hand books and then destroying them).
> one could also question the ethics of buying rare, second-hand books and then destroying them
I believe that is exactly what the parent post is doing
> So why are ai companies shredding books
Because a cartoon mouse's parent company used regulatory capture to extend copyright for decades.
I'm sure they're using whatever they can get their hands on, including Anna's archive. The business process of buying books to scan offers the opportunity to ingest media that hasn't yet been digitized, but also provides plausible legal deniability against copyright holders bringing litigation.
Anna's archive is missing books and papers.
I want to believe most of what they're missing is material held in national and public university libraries and is material specific to the country the books or papers come from. Most of their collections are not digitized and that's what I keep seeing, and then some books that are profession-specific.
They're basically rebuilding Google books, and keeping it private. That's privatizing all of humanity's knowledge. Then from what I see some other missing books are just rephrasings of other books, which can already be done with synthetic data using another LLM to generate it. I would assume a lot of those books are school text books as well, which are almost never digitized and are rephrasings of university text books. Maybe they are not able to generate synthetic data of sufficient quality and are resorting to human books that rephrase information instead
i would also assume, most of the cheap books are donated books from all over the world. that are imported into the us for resale as book lots. it would be similar to how donated clothes are packaged into lots and then resold to third world countries, which then sell them in fashion or clothes outlets. then, they really would be importing books from all over the world into the us for scanning and shredding for ai.
there is an amazing subplot in the book "service model" that describes a set of robot librarians doing exactly this kind of thing, plus sorting every bit.
Abandoning ethics doesn't mean you actively seek out unethical acts, it just means you only care about pragmatics. Limiting how overtly you break the law, copyright law in this case, is often pragmatic regardless of ethics.