Comment by TFNA

Tech companies are training off the shadow libraries, which have the bulk of English-language human writing in book form from the twentieth century, and that traditionally published stuff with an ISBN usually got a proper editor of some kind. So, it is hard to believe LLM companies don't have enough material; it's not as if they are limited to low-quality stuff on the web.