Comment by MichaelZuo

If GPT-3 et al. are producing results on par with "underpaid freelance (ESL) writer farm" then wouldn’t the next step be to focus only on better writing? i.e. books, newspapers, magazines, etc.

Obviously the corpus (perhaps 1 trillion to to 10 trillion words?) will be exhausted by a sufficiently large model so there’s an upper bound.