Comment by rhelz

Chomsky's worried about ChatGTP for largely the same reasons that Orwell was worried about NewSpeak. I hope this example from Orwell will help: "The octopus of global capitalism has sung its swan song."

Obviously, that sentence was just pasted together from previously written slogans. Metaphors which were once bright and powerful (the tentacles of an octopus squeezing the life out of its victim! Or the bittersweet beauty of a dying swan singing). Which is sinful enough, but the only reason such sentences can get written or published is obviously that the thought has died alongside the metaphors.

But that is how these LLMs work: take an input prompt, find what would likely have been said based on how we used to use language yesterday and in the past, and put it through a meat-grinding-style rephrasing machine until it is impossible to tell who originally said it so it sounds original.

The seduction is that ChatGTP is so damn useful. As people lean on it, and their ability to think and say new things atrophies, a mental sclerosis might set it. And--just because our own language facilities have degraded--we might be incapable of even realizing what has happened to us.