Comment by dofm

I think as much it is that people write by grappling for the right phrase to represent some inner feeling or concept, writing in part for themselves, whereas LLMs write always and only for an audience.

It’s much easier to understand this once you think about other generative forms. MidJourney never just sits down and draws for fun, so fun never informs its art (only the outward appearance of others’ fun, separate from the fun itself). Suno doesn’t waste hours trying to find riffs on a guitar, so its output is never informed by the direct joy of getting it right. Its music is never optimised for playability on a particular guitar with a scratchy seventh fret and a too-high action. Neither Midjourney nor Suno have evolved their styles due to short-sightedness or carpal tunnel.

If you had a human writer who over a long career only ever wrote articles from an outline given to them by someone else, and you had all the outlines and all the resulting articles from those outlines, and you could train an LLM to generate an article from an outline, it still would not be kicking itself frustrated by an inelegant phrase in a prior article, it would not avoid certain phrases out of a passive aggressive reaction to some editor’s note, it would not ever just rush an article because everyone is gathering at the pub, and it would not choose an analogy just to rub the author of a bitchy critical letter to the editor the wrong way. An LLM could not “subtweet”. It could not write a series of articles hoping one important person will spot that they are auditioning for a job.

Creators have unseen, undocumented influences and motivations that inform their work over a long period. I don’t mean to say that these individual influences can be reliably detected in individual pieces of work. I do mean to say that I think their broad absence tends to be felt in LLM writing. As readers we develop an affinity for writers as much as for their writing, and we do this in part because we deduce things about them.