Comment by TeMPOraL
15 days ago
My pet theory is that this was happening, because labs naturally worked to source highest quality data sources they could get. Even if they initially slurped everything indiscriminately, the obvious next step is to sort them by "quality", and bias towards the high end while trimming from low end.
So obviously, government documents, top publications from liberal arts scholars and STEM > Wikipedia > Reddit (and within, r/AskHistorians > r/all, etc.). The first group is the one most likely to (over)use correct / fancy typography.
I have a related pet theory: the telltale sign of AI writing is being too good for the context it's found in, in terms of vocabulary and style as well as structure.
What is jarring about this is precisely the quality - we associate certain styles with important (or self-important) venues, so it's uncanny to suddenly see random shitposts, copywriting spam, or regular chat replies look like work of NYT bestselling authors or legal scholars - it feels wrong in places we expect basic language, grug-brain reasoning, and lots of typos.
Also, what makes the above much more noticeable, is that AI systems don't have cross-session stylistic memory. From the POV of an LLM, that perfectly fitting, masterfully crafted starter is the first and only time it used it. From the user's POV, it's the tenth time they're seeing it in the span of an hour.
There's also no voice. If I get a text from a friend, I can hear them. Their tones, their mannerisms. I can't hear Claude because it's not a person that I've met and interacted with.
Would that not also apply to most books, newspapers, and internet comments?
People use language in their own distinct fashion. To the point where you can often figure out the writer from a long enough sample. Newspapers and internet comments end up a blend of different voices but in visually distinct boxes.
LLM intentionally randomize output to cover up all the plagiarism, but this largely destroys the consistency that cause writers to feel unique. Larger patterns still emerge but it just feels stale.
4 replies →
can't say the same for newspapers or internet comments, but when reading books there's usually a voice or a "vibe", and if there isn't one then it's just an unreadable book
AI writing style comes from post-training, which is entirely under the control of the labs. It's not copying it from any public sources.
I didn't say it's copying anything. It's learning styles - all of them - in pre-training. Post-training is just about giving it the right defaults.
Doesn't change my point anyway - even if post-training, labs naturally want the LLM to come off as smart (well, except for Grok). Post-training is where they "untrained" em-dashes and such, due to complaints of users.
I'd heard an interesting theory that in training on customer chats there's a lot of interactions like this:
Agent: _wall of text_
User: yeah sounds great, do it
Which has the predictable outcome of biasing towards wall of text responses.