Comment by skissane
5 days ago
Never seen that tool before, but it has an obvious bug: paste in markdown-formatted text with `-` bullet points, and it told me a sign of AI writing was: `Repeated sentence openers · 3 sentences opening “-”`
Now, the text I pasted in so happened to really be AI-generated, but I don't think using markdown bulleted lists is a reliable sign of AI-generation.
Interesting! I tried it with unordered Markdown lists (both nested and single-level) and I couldn't reproduce it. Would you mind sharing a small repro? In some ways I guess the brittleness is to be expected since IIUC, it uses regexes under the hood.
Here is my input: https://gist.github.com/skissane/dfb7a569968224ec98cda757371...
To be clear, the issue is not that it says that text is AI-generated – that text is indeed AI-generated. The issue is the detection reason it gives:
Repeated sentence openers · 3 sentences opening “-”
A nested markdown list is not a reliable sign of AI-generation, humans were using them long before any AI was.
Similarly, its second "AI tell" is:
Echoing sentence runs · 2 sentences echoing “google sso auto create”
Two sentences beginning with the `GOOGLE_SSO_AUTO_CREATE_` is not a sign of AI generation, it is a sign of quoting code
Only the third match is a genuine "sign of AI generation"
A reliable sign of AI writing is the middot in the tool's output. But that is to be expected, given the tool's author.
A coworker assumed I shared AI slop because I wrote about 200-300 words of text with nice formatting: different heading text levels, monospace formatting for variable names, used different highlight colours, etc.
Can’t even write pleasant technical documentation by hand after memorizing hotkeys without being falsely accused…