Comment by akersten
7 hours ago
There's an underappreciated comment in the other thread about SynthID and OpenAI [0] that captures what (IMO) the hacker ethos on this should be. We care about privacy, we should not accept tools that barcode our every digital move. (note that the counter of "well, they don't do that yet" is not particularly convincing)
Building a tool that tries (and probably fails) to remove the watermark (due to the arms race that large corporate machines will win) is tacitly accepting the barcode. The hacker ethos should be, first and foremost, to run open source models locally without relying on a corporation.
>due to the arms race that large corporate machines will win
Much like how the entirety of Hollywood, book publishers, academic publishers, and game developers have won against piracy despite being some of the largest corps on earth and dedicating untold billions to the issue over the past 30 years?
They won the long game. Everything is rented and DRM now. Very little of what most people buy digitally is truly owned.
2 replies →
They have a finite # of employees, a finite budget, and a finite amount of time.
Hobbyists do not. ROI is not a factor.
6 replies →
What? Some nerds on private trackers and kids on 123movies or whatever is not piracy winning by any material stretch.
Yes. Winning against piracy doesn't mean you completely eliminate piracy. It means you scare enough people into not doing it and make it a bit harder to do for others.
Losing to piracy would see companies like Netflix and Spotify not thriving.
1 reply →
> [fighting against the system] is tacitly accepting the barcode.
I don't really see it. I think it's important to win on both fronts.
Especially as the open weight models are really generated by corporates, and they could stop releasing them at any time.
3 replies →
Fighting within the system is accepting the system.
> No use messing with Google's watermark, fellas. Go do something else that's 100x harder instead.
> works for Google
Gee, I wonder why...
This is the “instead of using seatbelts, we should invest in trains” argument.
[dead]
Accepting blindly destroying the concept of thruth should not be the hacker ethos either.
It's already possible to lie with text. Pixels are pixels. If we can't blindly believe pixels to show the truth, we will be simply back to the pre-photography era which managed to have a concept of truth regardless.
When could you ever trust pixels?
1 reply →
It either works reliably or it doesn't; if it doesn't, it's better that everybody be clear about that.
Not necessarily. Knowing an image for sure is fake has value, even if you can’t guarantee the reverse is true.
Fair enough. While I would kind of wish AI could be reliably detected, deep down I know this is impossible and it would be pretty bad if we had, say, a prosecution that succeeded because "this 'provably-non-AI' photo places you at the scene of the crime" because only a few underground people know how to remove a watermark.
1 reply →
It's best for privacy not to do this in the first place because:
- Watermarks are optional by AI provider so bad actors will circumvent by using another provider
- GH project proves watermarks can be removed
Given these, trying to ensure "truth" is a futile effort unfortunately, and watermarking only gives companies advantage to violate privacy
Nobody said that?
Saying that watermarking fake things is bad kinda strongly implies it
1 reply →
The concept of truth? A bit overblown don't you think? Because some guy can make a realistic looking fake videos that destroys the "concept" of truth? How?
Stalin had no issues photoshopping images almost 100 years ago.
Generating realistic video of arbitrary things and people at scale is quite a bit of a different game than retouching photos
Stalin had all the resources imaginables at his disposal.
Now Nancy, a tech-phobic waitress who has a grudge against her coworker can make up an entire scenario with one prompt and her colleagues might blindly believe her.
Let's not pretend they're the same thing.
Gen AI is inevitable. Watermarking is likely futile. But in my opinion it is still very important to discuss how, as a society, we're going to live in a post-truth world now that anybody can, IN SECONDS, not only fabricate a story but also spread it to thousands of people through their social media.
7 replies →
A good example why fake images are bad.
Do you want to make it easier for the next Stalin?
1 reply →
I'm pretty sure watermarking is (or soon will be) a requirement for AI generated images in software used in the EU, as part of their regulations for AI transparency.
Of course. Regulations are the EUs primary output these days! Anywhere else they’re just sparkling suggestions.
If i had a dollar for every time an American cried about literally any non-US jurisdiction having an iota of effect on them I could quit my job and leave this terrible website forever.
1 reply →
It's not "every digital move" it's the photos you ask them to create. If you care about privacy use a local model
Its what happens when people in power are paranoid dark-triad types and want to be able to catch anyone who threatens their power and stick it to them..
It already happened with Trump claiming any unfavorable content of his administration is "AI-generated" as a defense to dismiss real, unedited media. He literally said, “If something happens that’s really bad, maybe I’ll have to just blame AI.”
ie the video of garbage being thrown out the windows that his team already confirmed was real:
https://www.kptv.com/2025/09/03/trump-says-video-showing-ite...
Also the Lincoln Project video footage him him stumbling while walking and over his words: https://www.forbes.com/sites/mattnovak/2023/12/04/donald-tru...
Do we care about truth?
Without truth freedom and privacy are endangered too.
The other comment talks about laws that can already handle that. How if images, video and audio aren’t reliable proof anymore?
Maybe we do care about truth, freedom and privacy but the majority of rest of society will happily accept any T&Cs just to get access to whatever the next digital sliced pan is and as for truth and accountability, if they were two sides of the same coin on the ground people wouldn't bend down to pick it up as possesing it looks too much like responsibility and inconvenience.
The watermarking should be on those things we want to verify as something that was not generated or manipulated. Something you'd add to, for instance, cameras. Putting them on the generated/manipulated is backwards as you can never get every model to watermark.
That model is equally bad though. Given that you're writing this in a discussion about gen AI watermarks, how in the world did you come up with the idea that Gen AI wouldn't be able to add a watermark?
I think you'll have to clarify the cause and effect of that a bit.
Also note that people have been falling for obviously watermarked videos already.
And even if they weren't, wouldn't that just make them more gullible towards non-watermarked models?
The human ethos should be to never be misleading about the origin and truth of any content you create, forward, or pass on. If we care about honesty we should jail anyone who does so.