Comment by jerf
1 hour ago
"What would it even look like to sign ChatGPT outputs? There’s no artifact to pass around."
It would look like a lot of little signatures on little bits of text, and then larger signatures on a collection of those chunks once the larger chunk exists. It's not that hard. It's just a lot of signatures.
But this scheme could only ever prove that this bit of text was made by a given AI, and validate anything else ever included in the signature hasn't been tampered with. It's not hard to work up a scheme that proves (within reason) a text was generated no earlier than some date by incorporating some sort of information that could only have been known at that date so that could be validated. But this isn't even a step in the direction of proving that something was made by a human. And that's assuming the private keys stay private, which is its own tricky problem. If a private key ever leaks anything signed with it becomes invalidated.
> And that's assuming the private keys stay private, which is its own tricky problem. If a private key ever leaks anything signed with it becomes invalidated.
We don't appear to be talking about cryptographic signing here. (That would never work for the problem because everyone expects unsigned text anyway.) We're talking about:
> It’s basically a text steganography problem (concealing a secret code), made more difficult because the plaintext cannot be arbitrarily manipulated.
As in, trying to force ChatGPT output to contain intentionally crafted ChatGPT-specific LLMisms that a human is unlikely to imitate, even one who reads a lot of ChatGPT output.
> It would look like a lot of little signatures on little bits of text, and then larger signatures on a collection of those chunks once the larger chunk exists. It's not that hard. It's just a lot of signatures.
But how is this implemented? It's a few lines of code to implement a basic "identify and strip/replace any non printable ascii, unicode ..." or whatever. A screenshot/OCR will also do this.
SO at the end of the day you're left with some dumb rules like "you used `load-bearing` more than once per 500 words, that's AI!"
To quote the original article's context more fully:
"The output of chat tools (and most of the output of AI agents) is not containerized text, but plain old regular text, and so can’t be signed. What would it even look like to sign ChatGPT outputs? There’s no artifact to pass around."
I'm reacting to the idea that "plain old regular text" can't be "signed" because they aren't "files". I'm observing that you can sign a stream of text, and probably other metadata, no problem. To my reading this really is about signing and not stegonographic watermarking, so we're in a context where for some reason the users in question want to carry the certificate of generation by AI and so the fact that this is trivially strippable isn't the issue at hand.
I read it this way because it seems to me clear that it isn't any particularly harder to do the stenographic stuff on a stream than a file (per zahlman's comment), so it only makes sense to be talking about this if we are actually talking about signing.
It's done with a lot more subtlety and embedded directly into the content, no Unicode shenanigans. The basic idea is just to break token generations where the probability is nearly tied in favor of the side that matches the secret key. With a long enough text block you can be statistically certain if the generation was using the key. From another comment: https://johnjwang.com/post/2026/08/12/how-claude-watermarkin...