Comment by iamflimflam1
5 hours ago
I did some experiments with very simple AI detection. You can get a very long way with simple ngram probabilities.
https://www.atomic14.com/2026/08/18/detecting-claude-with-le...
It’s very hard to make reliable though. Different models have different characteristics and you can prompt your way out of being detected.
This is exactly how we should approach problems. Not through just throwing more resources at it (fighting GPU compute with GPU compute) but by being clever.
Thanks a lot for sharing!
Feeding it samples of a long-going conversation with Gemini 3.1 Pro is interesting. The first message seems to get flagged instantly, but later ones sometimes pass as human. Or at least more human-ish.
If I read the blogpost correctly, you've only "trained" on prompt<->response and not interactive sessions?
Interesting! Amusingly, if I feed that detector this blog post, it identifies it as confidently robot (97 out of 100 test passages). And running through my last five blog entries, they are all over the map, with three deemed at least "likely robot." Looking further back in time (and taking a somewhat random example), a blog entry from 2008, "Concurrency's Shysters"[0], is also deemed as similarly confidently robot (also 97 out of 100); do you expect this high a false positive rate?
[0] https://bcantrill.dtrace.org/2008/11/03/concurrencys-shyster...
It really depends - it’s trained on fairly limited data (things I could generate from Claude opus 5 and ChatGPT (pre-Astra).
It’s now quite hard to get non AI training data…