Comment by theoldgreybeard
1 day ago
The interesting tidbit here is SynthID. While a good first step, it doesn't solve the problem of AI generated content NOT having any kind of watermark. So we can prove that something WITH the ID is AI generated but we can't prove that something without one ISN'T AI generated.
Like it would be nice if all photo and video generated by the big players would have some kind of standardized identifier on them - but now you're left with the bajillion other "grey market" models that won't give a damn about that.
> now you're left with the bajillion other "grey market" models that won't give a damn about that.
Exactly. When the barrier to entry for training a okay-ish AI model (not SOTA, obviously) is only a few thousand compute hours on H100s, you couldn't possibly hope to police the training of 100% of new models. Not to mention that lots of existing models are already out there are fully open-source. There will always be AI models that don't adhere to watermark regulations, especially if they were created a country that doesn't enforce your regulations.
You can't hope to solve the problem of non-watermarked AI completely. And by solving it partially by mandating that the big AI labs add a unified watermark, you condition people to be even more susceptible to AI images because "if it was AI, it would have a watermark". It's truly a no-win situation.
Some days it feels like I'm the only hacker left who doesn't want government mandated watermarking in creative tools. Were politicians 20 years ago as overreative they'd have demanded Photoshop leave a trace on anything it edited. The amount of moral panic is off the charts. It's still a computer, and we still shouldn't trust everything we see. The fundamentals haven't changed.
> It's still a computer, and we still shouldn't trust everything we see. The fundamentals haven't changed.
I think that by now it should be crystal clear to everyone that it matters a lot the sheer scale a new technology permits for $nefarious_intent.
Knives (under a certain size) are not regulated. Guns are regulated in most countries. Atomic bombs are definitely regulated. They can all kill people if used badly, though.
When a photo was faked/composed with old tech, it was relatively easy to spot. With photoshop, it became more complicated to spot it but at the same time it wasn't easy to mass-produce altered images. Large models are changing the rules here as well.
I think we're overreacting. Digital fakes will proliferate, and we'll freak out bc it's new. But after a certain amount of time, we'll just get used to it and realize that the world goes on, and whatever major adverse effects actually aren't that difficult to deal with. Which is not the case with nuclear proliferation or things like that.
The story of human history is newer generations freaking about progress and novel changes that have never been seen before. And later generations being perfectly okay with it and adapting to a new style of life.
5 replies →
> a new technology permits for $nefarious_intent
But people with actual nefarious intent will easily be able to remove these watermarks, however they're implemented. This is copy protection and key escrow all over again - it hurts honest people and doesn't even slow down bad people.
> Knives (under a certain size) are not regulated. Guns are regulated in most countries. Atomic bombs are definitely regulated
I don’t think this is a good comparison: knives are easy to produce, guns a bit harder, atomic bombs definitely harder. You should find something that is as easy to produce as a knife, but regulated.
3 replies →
Politicians absolutely were doing this 20-30 years ago. Plenty of folks here are old enough to remember debates on Slashdot around the Communications Decency Act, Child Online Protection Act, Children's Online Privacy Protection Act, Children's Internet Protection Act, et al.
https://en.wikipedia.org/wiki/Communications_Decency_Act
It’s annoying how effective “for the children” is. That peiole really just turn off their brains for that.
1 reply →
I suspect watermarking ends up being a net negative, as people learn to trust that lack of a watermark indicates authenticity. Propaganda won’t have the watermark.
Easy to say until it impacts you in a bad way:
https://www.nbcnews.com/tech/tech-news/ai-generated-evidence...
> “My wife and I have been together for over 30 years, and she has my voice everywhere,” Schlegel said. “She could easily clone my voice on free or inexpensive software to create a threatening message that sounds like it’s from me and walk into any courthouse around the country with that recording.”
> “The judge will sign that restraining order. They will sign every single time,” said Schlegel, referring to the hypothetical recording. “So you lose your cat, dog, guns, house, you lose everything.”
At the moment, the only alternative is courts simply never accept photo/video/audio as evidence. I know if I were a juror I wouldn't.
At the same time, yeah, watermarks won't work. Sure, Google can add a watermark/fingerprint that is impossible to remove, but there will be tools that won't put such watermarks/fingerprints.
Testimony is evidence. I don't think most cases have any physical evidence.
1 reply →
In the past, and maybe even to this very day - all color printers print hidden watermarks in faint yellow ink to assist with forensic identification of anything printed. Even for things printed in B&W (on a color printer).
https://en.wikipedia.org/wiki/Printer_tracking_dots
Yes, can we not jump on the surveillance/tracking/censorship bandwagon please?
Unless they've recently changed it, Photoshop will actually refuse to open or edit images of at least US banknotes.
You do know that every color copier comes with the ability to identify US currency and would refuse to copy it? And that every color printer leaves a pattern of faint yellow dots on every printout that uniquely identifies the printer?
Is this something strictly with the US currency notes or is the same true for other countries currency as well?
1 reply →
And that's not a good thing.
7 replies →
Try photocopying some US dollar bills.
HN is full of authoritarian bootlickers who can't imagine that people can exist without a paternalistic force to keep them from doing bad things.
I'm sure Apple will roll something out in the coming years. Now that just anyone can easily AI themselves into a picture in front of the Eiffel tower, they'll want a feature that will let their users prove that they _really_ took that photo in front of the Eiffel tower (since to a lot of people sharing that you're on a Paris vacation is the point, more than the particular photo).
I bet it will be called "Real Photos" or something like that, and the pictures will be signed by the camera hardware. Then iMessage will put a special border around it or something, so that when people share the photos with other Apple users they can prove that it was a real photo taken with their phone's camera.
This exists, the standard is called C2PA, Google added support for it in the Pixel 10. I was surprised and disappointed that Apple didn’t add support for it in the most recent iPhone! A few physical cameras are starting to support it too (https://yawnbox.eu/blog/c2pa-camera/)
Does anyone other than you actually care about your vacation photos?
There used to be a joke about people who did slideshows (on an actual slide projector) of their vacation photos at parties.
> a real photo taken with their phone's camera
How "real" are iPhone photos? They're also computationally generated, not just the light that came through the lens.
Even without any other post-processing, iPhones generate gibberish text when attempting to sharpen blurry images, they delete actual textures and replace them with smooth, smeared surfaces that look like a watercolor or oil paintings, and combine data from multiple frames to give dogs five legs.
Don’t be a pedant. You know very well there is a big different between a photo taken on an iPhone and a photo edited with Nano Banana.
this already exists. its called 35mm film camera.
Can't wait for a machine printing images on film by exposing it with a laser.
The incentive for commercial providers to apply watermarks is so that they can safely route and classify generated content when it gets piped back in as training or reference data from the wild. That it's something that some users want is mostly secondary, although it is something they can earn some social credit for by advertising.
You're right that there will existed generated content without these watermarks, but you can bet that all the commercial providers burning $$$$ on state of the art models will gradually coalesce around some means of widespread by-default/non-optional watermarking for content they let the public generate so that they can all avoid drowning in their own filth.
If there was a standardized identifier, there would be software dedicated to just removing it.
I don't see how it would defeat the cat and mouse game.
It doesn't have to be perfect to be helpful.
For example, it's trivial to post an advertisement without disclosure. Yet it's illegal, so large players mostly comply and harm is less likely on the whole.
You'd need a similar law around posting AI photos/videos without disclosure. Which maybe is where we're heading.
It still won't prevent it, but it would prevent large players from doing it.
I don't think it will be easy to just remove it. It's built into the image and thus won't be the same every time.
Plus, any service good at reverse-image search (like Google) can basically apply that to determine whether they generated it.
There will always be a way to defeat anything, but I don't see why this won't work for like 90% of cases.
> I don't think it will be easy to just remove it.
No, but model training technology is out in the open, so it will continue to be possible to train models and build model toolchains that just don't incorporate watermarking at all, which is what any motivated actor seeking to mislead will do; the only thing watermarking will do is train people to accept its absence as a sign of reliability, increasing the effectiveness of fakes by motivated bad actors.
> I don't think it will be easy to just remove it.
Always has been so far. You add noise until the signal gets swamped. In order to remain imperceptible it's a tiny signal, so it's easy to swamp.
It's an image. There's simply no way to add a watermark to an image that's both imperceptible to the user and non-trivial to remove. You'd have to pick one of those options.
4 replies →
You could probably just stick your image in another model or tool that didn't watermark and have it regenerate the image as accurately as possible.
1 reply →
It would be like standardizing a captcha, you make a single target to defeat. Whether it is easy or hard is irrelevant.
There will be a model trained to remove synthids from graphics generated by other models
This is what C2PA is trying to do: https://c2pa.org/
This is what the SynthID signature looks like on Nano Banana images https://www.reddit.com/r/nanobanana/comments/1o1tvbm/
And if it can be seen like that, it should be removeable too. There are more examples in that thread.
> more examples in that thread
Some supposition: A Fourier amplitude image should show that pattern as peaks at a certain angle/radius location. The exact location may be part of the identification scheme. Running peak finding on the Fourier image and then zeroing out the frequencies in the peak should remove the pattern. Modeling the shape of the peak would allow mimicking the application of a legit SynthID signature.
If anyone tries/tried this already, I'd love to see the results.
It solves some problems! For example, if you want to run a camgirl website based on AI models and want to also prove that you're not exploiting real people
> It solves some problems! For example, if you want to run a camgirl website based on AI models and want to also prove that you're not exploiting real people
So, you exploit real people, but run your images through a realtime AI video transformation model doing either a close-to-noop transformation or something like changing the background so that it can't be used to identify the actual location if people do figure out you are exploiting real people, and then you have your real exploitation watermarked as AI fakery.
I don't think this is solving a problem, unless you mean a problem for the would-be exploiter.
Your use case doesn't even make sense. What customers are clamoring for that feature? I doubt any paying customer in the market for (that product) cares. If the law cares, the law has tools to inquire.
All of this is trivially easy to circumvent ceremony.
Google is doing this to deflect litigation and to preserve their brand in the face of negative press.
They'll do this (1) as long as they're the market leader, (2) as long as there aren't dozens of other similar products - especially ones available as open source, (3) as long as the public is still freaked out / new to the idea anyone can make images and video of whatever, and (4) as long as the signing compute doesn't eat into the bottom line once everyone in the world has uniform access to the tech.
The idea here is that {law enforcement, lawyers, journalists} find a deep fake {illegal, porn, libelous, controversial} image and goes to Google to ask who made it. That only works for so long, if at all. Once everyone can do this and the lookup hit rates (or even inquiries) are < 0.01%, it'll go away.
It's really so you can tell journalists "we did our very best" so that they shut up and stop writing bad articles about "Google causing harm" and "Google enabling the bad guys".
We're just in the awkward phase where everyone is freaking out that you can make images of Trump wearing a bikini, Tim Cook saying he hates Apple and loves Samsung, or the South Park kids deep faking each other into silly circumstances. In ten years, this will be normal for everyone.
Writing the sentence "Dr. Phil eats a bagel" is no different than writing the prompt "Dr. Phil eats a bagel". The former has been easy to do for centuries and required the brain to do some work to visualize. Now we have tools that previsualize and get those ideas as pixels into the brain a little faster than ASCII/UTF-8 graphemes. At the end of the day, it's the same thing.
And you'll recall that various forms of written text - and indeed, speech itself - have been illegal in various times, places, and jurisdictions throughout history. You didn't insult Caesar, you didn't blaspheme the medieval church, and you don't libel in America today.
> What customers are clamoring for that feature? If the law cares, the law has tools to inquire.
How can they distinguish from real people exploited to AI models autogenerating everything?
I mean right now this is possible, largely because a lot of the AI videos have shortcomings. But imagine in 5 years from now on ...
2 replies →
You have to validate from the other direction. Let CCD sensors sign their outputs, and digital photo-editing produce a chain of custody with further signatures.
Maybe zero knowledge proofs could provide anonymity, or a simple solution is to ship the same keys in every camera model, or let them use anonymous sim-style cards with N-month certificate validity. Not everyone needs to prove the veracity of their photos, but make it cheap enough and most people probably will by default.
SynthID has been in use for over 2 years.
Regardless of how you feel about this kind of steganography, it seems clear that outside of a courtroom, deepfakes still have the potential to do massive damage.
Unless the watermark randomly replaces objects in the scene with bananas, these images/videos will still spread like wildfire on platforms like TikTok, where the average netizen's idea of due diligence is checking for a six‑fingered hand... at best.
I don't understand why there isn't an obvious, visible watermark at all. Yes, one could remove it but let's assume 95% of people don't bother removing the visible watermark. It would really help with seeing instantly when an image was AI generated.
It would be more productive for camera manufacturers to embed a per-device digital signature. Those care to prove their image is genuine could publish both pre and post processed images for transparency.
Reminder that even in the hypothetical world where every AI image is digitally watermarked, and all cameras have a TPM that writes a hash of every photo to the blockchain, there’s nothing to stop you from pointing that perfectly-verified camera at a screen showing your perfectly-watermarked AI image and taking a picture.
Image verification has never been easy. People have been airbrushed out of and pasted into photos for over a century; AI just makes it easier and more accessible. Expecting a “click to verify” workflow is unreasonable as it has ever been; only media literacy and a bit of legwork can accomplish this task.
Competent digital watermarks usually survive the 'analog hole'. Screen-cam resistant watermarks have been in use since at least 2020, and if memory serves, back to 2010 when I first starting reading about them, but I don't recall what it was called back then.
I just tried asking Gemini about a photo I took of my screen showing an image I edited with Nano Banana Pro... and it said "All or part of the content was generated with Google AI. SynthID detected in less than 25% of the image".
Photo-of-a-screen: https://gemini.google.com/share/ab587bdcd03e
It reported 25-50% for the image without having been through that analog hole: https://gemini.google.com/share/022e486fd6bf
1 reply →
Take this a step further and it'll be a personal identifying watermark (only the company can decode). Home printers already do this to some degree.
yeah, personally identifying undetectable watermarks are kindof a terrifying prospect
It is terrifying, but inevitable. Perhaps AI companies flooding the commons with excrement wasn't the best idea, now we all have to suffer the consequences.
This watermarking ceremony is useless.
We will always have local models. Eventually the Chinese will release a Nano Banana equivalent as open source.
Qwen-Image-Edit is pretty good already: https://simonwillison.net/2025/Aug/19/qwen-image-edit/
Qwen won the latest models round last month…
https://generative-ai.review/2025/09/september-2025-image-ge... (non-pro Nano Banana)
> We will always have local models.
If watermarking becomes a legal mandate, it will inevitably include a prohibition on distributing (and using and maybe even possessing, but the distribution ban is the thing that will have the most impact, since it is the part that is most policable, and most people aren't going to be training their own models, except, of course, the most motivated bad actors) open models that do not include watermarking as a baked-in model feature. So, for most users, it'll be much less accessible (and, at the same time, it won't solve the problem.)
I don't see how banning distribution would do anything: distributing pirated games, movies, software is banned in most countries and yet pirated content is trivial to find for anyone who cares.
As long as someone somewhere is publishing models that don't watermark output, there's basically nothing that can stop those models from being used.
We need to be super careful with how legislation around this is passed and implemented. As it currently stands, I can totally see this as a backdoor to surveillance and government overreach.
If social media platforms are required by law to categorize content as AI generated, this means they need to check with the public "AI generation" providers. And since there is no agreed upon (public) standard for imperceptible watermarks hashing that means the content (image, video, audio) in its entirety needs to be uploaded to the various providers to check if it's AI generated.
Yes, it sounds crazy, but that's the plan; imagine every image you post on Facebook/X/Reddit/Whatsapp/whatever gets uploaded to Google / Microsoft / OpenAI / UnnamedGovernmentEntity / etc. to "check if it's AI". That's what the current law in Korea and the upcoming laws in California and EU (for August 2026) require :(
I don't believe that you can do this for photography. For AI-images, if the embedded data has enough information (model identification and random seed), one can prove that it was AI by recreating it on the fly and comparing. How do you prove that a photographic image was created by a CCD? If your AI-generated image were good enough to pass, then hacking hardware (or stealing some crypto key to sign it) would "prove" that it was a real photograph.
Hell, it might even be possible for some arbitrary photographs to come up with an AI prompt that produces them or something similar enough to be indistinguishable to the human eye, opening up the possibility of "proving" something is fake even when it was actually real.
What you want just can't work, not even from a theoretical or practical standpoint, let alone the other concerns mentioned in this thread.
It solves a real problem - if you have something sketchy, the big players can repudiate it, the authorities can more formally define the black market, and we can have a ‘war on deepfakes’ to further enable the authorities in their attempts to control the narratives.
Labelling open source models as "grey market" is a heck of a presumption
Every model is "grey market". They're all trained on data without complying with any licensing terms that may exist, be they proprietary or copyleft. Every major AI model is an instance of IP theft.
It's why I used "scare quotes".
I asked Gemini "dymamic view" how SynthID works: https://gemini.google.com/share/62fb0eb38e6b