Comment by stymaar

> Over-using em-dash or whatever isn't the thing that maximizes engagement.

It's the thing that minimizes the loss during the RLHF phase, and the RLHF phase is the one that's aimed at maximizing engagement (it's literally trained on that).

> what happens to the actual humans whose writing style is a close match for what a given generation of LLMs output?

If a human, for instance because its writing gets polluted by reading too much AI slop, matches the style of an LLM closer than a certain threshold, then his own writing is going to be flagged as well. Whether it's an actual problem or merely a theoretical one is an open question. (unlike OpenAI and Anthropic, humans writers do have an incentive to avoid being flagged as AI).

> And, what stops LLMs from using a different style when someone wants to fool the classifier?

In theory: nothing. In practice if you fine-tune your own model: nothing. In practice with commercial models: the interests of the model making company.

> And, what stops LLMs from using a different style when someone wants to fool the classifier?

Websites have pretty much stopped using ad-blocker-blockers, it seems that it's not a fight worth fighting for them. Does that mean that ad-blockers are useless?

Most people don't even care about ads, I don't think they care about slop either, that's why there's slop posts and obnoxious websites that are unreadable without an ad blocker. A slop blocker used by 10-20% of the internet users wouldn't change the calculation more than ad blockers did.