Comment by echelon
6 hours ago
This is very similar to audiowmark
https://github.com/swesterfeld/audiowmark
You can stuff per-item database unique IDs, user IDs, geohashes, and other nefarious things inside.
We need to protest this LOUDLY.
Our devices are being locked down, we're having attestation and trusted computing forced on us, the internet all over the world is undergoing age verification with full ID verification.
Just because this is on "ai images" today doesn't mean it won't be on all images - screenshots, your camera reel, etc. - in the fullness of time.
This is scary.
These are the tools of 1984. They've been boiling the water slowly, but in the last year things have really started to pick up pace. Please push back. Loudly.
Everyone at Google and OpenAI working on this: WHAT THE FUCK ARE YOU DOING. STOP.
We have laws and mechanisms to prevent revenge porn, CSAM, defamation, etc. They are robust and can be made even stronger. We do not need to sacrifice the security of our privacy and our speech to fight imagined harms when the real danger is turning into an authoritarian society.
The point of SynthID is to make generated images identifiable, in an attempt to prevent 1984-esque situations where you can't believe your eyes and ears. Applying it to screenshots and camera output defeats its only purpose.
If the powers-that-be want to enforce age verification, watermarking camera output is not the correct technology to do so. It would be something like HDCP, where camera manufacturers are given keys and a whole trusted media path is built so that the relying party can cryptographically enforce that a trusted camera is being used to capture live images.
> The point of SynthID is to make generated images identifiable, in an attempt to prevent 1984-esque situations where you can't believe your eyes and ears.
You can still use traditional methods to manipulate images, too, so I don't think a "does not contain SynthID watermark" means you can trust that image more. In the other hand, encoding a lot of personal and other information in the watermark (136 bit is a lot) that can not be easily removed and most of the people are unaware of it seems really an 1984-like dystopia.
You have missed the point by such a wide margin that I have to wonder if it wasn't intentional.
The same techniques used here can be applied in other domains for other purposes. That would not "defeat its only purpose". The danger is the normalization of watermarking for [ insert good reason here ] with regulation eventually making it mandatory once everyone is accustomed to it. Rinse and repeat to gradually boil the frog.
We live in a world where nearly all printers already watermark everything they print with their serial number. It wouldn't be at all surprising if the next modernized variant of that technology encoded personal and contextual data tied to the user.
I'm going to save my protests for anyone trying to watermark real images.
Zero watermarks is a lot worse than semi-effective AI watermarks.
Is it? Given local models this delays the current cutting edge at any given time by what, 6 to 12 months at best?
Well the person I replied to seems to think it'll be at least semi-effective.
1 reply →
Most cameras already produce metadata. You can remove this metadata. Can you not also detect and remove watermarks?
The paper references some threat models they considered. They suggest someone might "possess paired information (both original and watermarked content)" and therefore be able to undo watermarking. Presumably it's fairly easy to get identity operations out of image APIs that would result in this situation. I'm not sure that addresses echelon's main concerns though.
The metadata is kept separately from the original data, and is, by design, modifiable and removable.
Watermark, by design, irreversibly modifies the original data, and is, by design, hard to remove without producing detectable artifacts (or rendering the data useless altogether).
In short, the answer is no.