Comment by saghm

Although you're referring to prompts given the Claude rather than the people attempting to recognize, it occurs to me that most of the discussion I've seen around people recognizing AI seems cover contexts where the reader is actively suspicious about whether content generated to begin with. Rather than a binary "is this text AI generated or not", I wonder if it would be harder for people to do a Coke/Pepsi style challenge where they're given two pieces of text where it's not guaranteed to be exactly one LLM-generated and one human-written, but they could both be from an AI or both be from a human.

Going further, I'm curious about whether people are mostly good at the case where they suspect most or all of the content from given "author" has the same amount of AI usage/prompting in generating it rather than the adversarial case where someone might usually use AI extensively and then try to slip by purely human written text (or vice-versa). I don't have a good sense of whether this is a threat model that actually matters, since maybe the heuristic of weeding out sources that are mostly AI-generated is enough for people who prefer to avoid that type of content, but I do think that changes the definition of what it means to be "good at recognizing AI" in a meaningful way. It seems plausible that disagreements about how easy it is to recognize AI content might be coming from two people assuming a different framing of the question that results in a different answer without realizing that's what they've done.