Comment by vidarh
1 day ago
Giving an LLM samples and tell it to apply the style in the sample works a lot better than just telling it to copy a style it may have seen, or a list of rules.
They do it well enough that it'd take really good output to beat.
They really don't.
If your goal is to say, write science fiction, their reversion to classic LLM-isms, is really distracting and is what makes people say from a glance that it was written by an LLM. You basically can't use them at the moment in any real "natural" long-form writing. Everyone will call "slop" pretty quickly on the current frontier models.
Rosmin's DFT paper is worth a read.
I have seen examples that shows otherwise, including from a client that tested it extensively by paying people who thought they were paid to help detect AI generated content. They did little more than what I described. It works very well. Some people still insist they are able to tell the difference, but in the tests I saw, people did little better than random chance.
Some of it you could probably tell with statistical analysis, but actualy people are far worse at judging whether content is AI generated than they think they are.
If you need to beat an AI testing tool, you need to do marginally more work than to stop people from recognising it, but not all that much.
The nature of it is that you don't "see" most of the stuff that is well done because few people want to talk about it.