Comment by ch_sm
2 hours ago
here‘s what i don‘t get about this whole discussion. AI companies already store all prompts and responses for future training.
just make an API that returns the string distance between a previously generated paragraph and the query?
that would sidestep this whole problem class.
regulators could even specify how that has to work.
what am i missing?
> AI companies already store all prompts and responses for future training.
They claim to only do this when you agree to this in your personal settings. Though Google does say they will train on it, unless you disable history and only use ephemeral chats. Anthropic has a setting for it and claims not to train by default.
Also it would be vulnerable to attacks and privacy problems. You could search for substrings about some suspected information, like "John Smith's medical records show advanced cancer" etc. Of course you'd have to guess the phrasing but still.
> AI companies already store all prompts and responses for future training.
They store some prompts and responses, not all, that's what you're missing.
Wouldn't this become pretty difficult to do at scale over time? Is there a way to compare the similarity in a database of responses without doing a search over every entry and comparing them? Because that would probably become pretty slow if literally every LLM output is saved and has to be scanned.
1. AI company buys and trains on an author’s book when it gets published, it’s now part of the training data.
2. Attacker asks the LLM for the opening sentences of the book, it goes into the generated responses database.
3. Later, a malicious user shows that the first few sentences of the authors book are identical to a previously generated response.
Ignoring the other technical hurdles of the idea...this problem you outlaid is solved with a timestamp in the database, right? You can easily prove if the prompt was before/after publishing date?
Local models?
Local models enable:
- watermark-free generation
- the stripping of watermarking from the output of SAAS models
Any discussion of watermarking is dead in the water in a world where we are permitted to have these things. I fear for the future.
Local models also enable:
- not being locked into a provider
- not being forced to have your prompts saved by a possible competitor
- an alternative to the duolopy we quickly see forming
- offline access
I fear for the future without local models, much more than the future with them, and would rather everyone had access to a local model than be certain we catch everyone copy+pasting LLM responses. Watermarking would be cool, but it's not worth losing local for.
1 reply →
Chill my dude. This is just a sane default which will catch normies copy pasting stuff from claude and chatgpt. It's good enough.
1 reply →