Comment by avaer
3 hours ago
That's missing the point. It's the distribution that's the "best", not the tokens. Then Anthropic comes in and makes the distribution something other than the best. The only saving grace is that Anthropic says it's not that bad.
Even so, I don't think it will stop here. Once this is in place, the next step is to put more and more identification into the AI generated content; might as well pack it in, it's not that bad, and if it is they won't admit it. There's no way for anyone to check. And your argument will still be technically correct but missing the point.
there's absolutely no reason to think Claude produces absolute best token distributions or that slight adjustments would be noticeable.
In fact we know it's not that good because we can often tell Claude's writing apart from human writing.
How do you know you picked the singular “best” set of tokens in your comment here?
Could it have been equal or better with slight variations in wording?
The slipper slop argument is too lazy to address directly. Argue A is bad because A, not because A might become B and you’ve got good arguments against B.
The watermark doesn’t change the distribution, only per-token selection. I think not understanding that is the source of most people’s FUD.
There's no difference between those two things. The distribution that matters is the distribution of tokens that are picked not the distribution of tokens the LLM model passed to the selector.
This comment disagrees with you: https://news.ycombinator.com/item?id=49324387
That comment merely says quality must be compromised. It doesn’t make it clear why that must be true. Empirical study seems to say that quality is not compromised, and looking at various proposed schemes, it seems intuitively true.