Comment by runako
2 days ago
This product does not even have a plausible theory of how it could work.
LLM-generated text does not carry a watermark or other identifying marks. The "theory" is that an LLM trained on human writing, to mimic human writing, can be distinguished from actual human writing in under 100 words.
Notably the first diagram on the research overview page (https://www.pangram.com/research/how-it-works) shows feedback for "misclassified human examples." This is a category error; Pangram will not find out when it has misclassified text in the wild, except in rare cases. Only the "licensed human-written text" in its training data can be used as feedback.
Scams like Pangram also cause real harms, mostly because laypeople do not understand that what is being offered is not possible. Pangram advertises 99.98% accuracy, and they pitch it as a tool for teachers and universities. Translated: if a college like University of Alabama rolled this out, you could expect ~40 students to have their lives upended by this snake oil, every year. (And how can one even prove that an allegation is false, that they did write a given text?) And this is the best case, using the number on Pangram's homepage.
Claude, a general-purpose model, can identify me, personally with stylometry in about 200 words. Is it really such a stretch to believe it’s possible for a special-purpose model to identify the ten or so main LLMs crossed with the fifty or so main styles people gave them write in?
> Claude, a general-purpose model, can identify me, personally
It cannot. this is a misconception. A human being is fully capable of writing 200 words that Claude will identify as not being written by them, because human beings are far more complex than Claude. A human can even choose to deliberately write in the style of a different human, even one who does not exist.
Sometimes people are writing instruction manuals; those are not written like their professional emails, which are not written like their personal emails. It is normal for people to be able to write in different voices/styles/etc. People code switch, people write for different audiences, people change over time, people are hurried or tired or sick, etc.
So no, an LLM cannot identify you uniquely in 200 words. But more to the point, most human communication is not in training sets. And Pangram has no way of course-correcting on the vast amount of data that is not in its training sets.
By comparison: the autonomous vehicle companies actually do need their products to verifiably work. So they also feed back human-analyzed data from real trips into their models. They can tell the model where it was right or wrong in the real world. This is the part Pangram cannot do! Pangram deployed at a university may be used to accuse a student of cheating, but then Pangram will never know for sure whether the text in question was written by a human or machine. The feedback loop is missing a critical step!
I have run the experiment like seven times now on different tracts of text, given to people who are not me. It’s a point of simple fact that Opus 4.7 can identify me when I’m writing fresh text in my voice. Does that change your conclusion if you were to grant it for the sake of argument (notwithstanding the fact that it’s actually true)? Or is your objection “it can identify one of your voices, the most commonly used one, and not the others” or something like that?