Comment by munk-a
21 days ago
Gosh there's a lot of corollary evidence pointing to his guilt but this is likely going to become more and more common and force the use of a lot more technical forensic resources.
Finding an original copy on a go-pro would likely be pretty compelling evidence but this (and the more scary politically centered questions like this) are why I wish we had a way to build a durable chain of custody into these technologies. It is infeasible from everything I've seen but it would be a big win for society.
> Gosh there's a lot of corollary evidence pointing to his guilt but this is likely going to become more and more common and force the use of a lot more technical forensic resources.
Nah. People who do something like this can't help but brag. They'll incriminate themselves in seconds voluntarily.
> I wish we had a way to build a durable chain of custody into these technologies
Do you? Consider for a moment all the dissidents and protestors who would be ensnared by their own devices then, with no "it was all ai" defense available?
I do - I think the videos and pictures that protestors smuggle out become less powerful if the state can dismiss them as fake and while most of us will remain skeptical of authority the more easy it is to fake something the more people you can convince of your falsehood.
I don't think the lack of a durable chain of custody really provides any protection - that protection needs to come from a strong legal system and social contract to protect whistleblowers. If you're thinking of, as an example, an Iranian smuggling out protest footage, they're already taking an extreme risk and have a state using numerous tools to try and track them down - but the lack of a durable chain gives a wide area of authorities to cast doubt on the truth.
I think your question is interesting to ponder and I think there are arguments in both directions - but my mind keeps coming back to the tank man photo being smuggled out of China and how much more difficult it would be in the modern world for a single image to carry such weight.
> I don't think the lack of a durable chain of custody really provides any protection - that protection needs to come from a strong legal system and social contract to protect whistleblowers.
That social contract is quite a bit of a hit&miss if you look at countries across the globe. Same for the strong legal system. Other concerns aside, does this not make the whole approach a non-starter?
1 reply →
I don't think the lack of a durable chain of custody really provides any protection
creating a fake video about a protest is going to upset those that want to hide the protest just as much.
Couldn't you just match the noise profile of the camera with the video?
How do you find the camera?
"I extracted and added the noise profile to the AI generated video with a goPro to make it look legitimate"
Put private key into every digital camera and hash/sign every frame. That private key is accompanied with manufacturer signature and can't be easily extracted. Mark all unsigned media as suspicious.
"and can't be easily extracted" is doing a lot of work there. People are very good at reverse-engineering. There would soon be a black market for 'clean' private keys that could be used to sign any video you want.
There's also always the "analog loophole". Display the AI-generated video on a sufficiently high-resolution / color gamut display and record it on whatever device has convenient specs for making the recording, then do some light post-processing to fix moire/color/geometry. This would likely be detectable, but could shift the burden of (dis-)proof to the defendant, who might not have the money for the expert witnesses required to properly argue the technical merits of their case.
More likely, the signing would have to use compression-resistant steganography, otherwise it's pretty easy to just remux/re-encode the video to strip the metadata.
There would also be a requirement for all playback to actually properly check the private keys and for all the parties involved in the process to be acting in good faith. Not only would you have a black market for individuals to scalp clean keys but you'd likely have nation states with interests putting pressure on local manufacturers to give them backdoors.
We'd probably hit a lot of that with SSL if it wasn't so unimportant from a political perspective[1]... but if the thing we were trying to secure is going to boost or damage some prominent politician directly then the level of pressure is going to be on a whole different scale.
1. And we might still have that corruption of SSL when it comes to targeted phishing attacks.
2 replies →
"can't easily be extracted" = "the number of people who can extract it is small but still non-zero"
And those people now have the power to put you in jail, by putting your camera's signature on illegal content.
You've also just made journalism 3 notches harder. Like documenting atrocities in, say, North Korea. Or for whistleblowers in your home steel mill run by a corporate slavedriver.
Oh. Also. Why are you choosing the camera side to put this on? Why not the AI side? Require watermarks and signatures for anything created in such a way…
…of course that has its own set of intractable problems.
Ideally, the keys would be per-manufacturer, like HDCP or (DVD-)CSS. Personally I don't think I'd love the idea of any kind of attestation like this, but if TPTB did implement it, I'd prefer a key per-manufacturer rather than each unit having its own unique signing key. We do have precedent, in the form of printer tracking dots, which were kept 'secret' from the public for 20 years. [0]
0: https://en.wikipedia.org/wiki/Printer_tracking_dots
That makes it easy to prove authenticity (has signature), but doesn’t solve the “prove it’s fake” problem.
Ideally, the prosecutor bears the burden of proof. We generally shouldn't impose systems that require defendants to prove a negative. I recognize that reality does not necessarily match this ideal.
1 reply →
Isn't that similar to this?
https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ...
That certainly won't be used to violate someone's privacy.
More surveillance and tracking won't be the solution.
Film cameras may be making a comeback.
[dead]
They got him for illegal BASE jumping and now they are going to get him for lying to the police about it too.