Comment by tpoacher
8 days ago
The part about a change in entropy was interesting.
Is there an easy way to get / compare the entropy of two passages? (e.g. to see if it has indeed dropped after gen ai manipulation).
And could this be used to flag AI-gen text (or at least, boring, soulless sounding text)
A lot of times, this entropy decay is found in semantic or stylistic space, which would be hard to detect (you couldn't use, e.g., Shannon Entropy). You'd have to ask questions like "is this point uninteresting?" or "is this trope overused?"--bad (human) writers are often guilty of this too, so that's why AI can be hard to detect.
Yes, still though, it might be interesting to be able to measure this easily.
E.g. when asking an AI to rephrase or summarise, if the entropy drops you might take that as a sign that it has eroded the style beyond what you might be willing to tolerate.
I wonder if the author had a particular method / tool in mind, or if they were just speaking abstractly.