Comment by teeray
4 hours ago
The problems are simply too great if an LLM detector has any false positives at all. Imagine how soul-crushing writing an entire dissertation by hand and having it rejected because some “good enough” LLM detector decides you write too much like an AI.
As I recall, a few years ago (in the era of first generation LLMs), a professor in Texas used an anti-plagiarism tool that flagged more than one-third of the class using AI in an exam, and used that finding to give them a failing grade.
If memory serves, one student objected strenously and ran the professor's own work (published 10 years earlier) into the same tool and it flagged that work as AI-generated.
EDIT: HN item from June 2023 https://news.ycombinator.com/item?id=36215823
Exactly. The more corporate and proper you tend to speak, the more likely it's to classify you as an LLM. It's like the classifiers want us to talk like trash at their current rate. This seems to be really problematic for ESL speakers/typers that may have been trained on a smaller, more proper subset of the language.
It depends on the application. Dissertation? Hell naw. Blog post? Absolutely, run it through that thing.
The problem is that ed-tech is absolutely ravenous for an LLM detector and would rather use snake oil than accept that it might not be possible.