Comment by gregorkas
3 days ago
I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.
This already exists: https://c2pa.org , https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ... . Support by camera makers is - spotty.
C2PA has lots of problems.
https://www.hackerfactor.com/blog/index.php?%2Farchives%2F10...
Doesn't this require a paid certificate? that effectively blocks open source software/hardware from implementing it.
And how do you fix the analog hole? Because if you can point your "verified" camera at a sufficiently high-resolution screen, we're worse off than when we started.
There are some techniques to detect recapture, e.g.: Moiré Pattern, Glare, JPEG Grid Artifacts, Channel Phase Shift, Screen Emission, Chromatic Aberration. If those are combined, the effort and cost to fake a photo rises significantly.
Yes, I’m more worried about the false confidence such technology could create. Implement an authenticity mechanism and it will be treated as truth. Powerful people will have the means to spoof photographic evidence.
You can have other sensors that tell you it's a screen, maybe require a Live Photo, maybe also upload to a third party service faster than generation is possible? In the end I think we'd end up somewhere like with cryptography: generating a real fake might be theoretically possible but it could be made prohibitively expensive to generate.
Depth sensor information.
Or just extract the certificate from the hardware you own.
That is presumably a very expensive endeavor. We already have hardware that attempts to mitigate this and while I think it's possible for the government it's certainly not trivial.
This is a "solved" problem. Vendors whose keys are extractable get their licenses revoked. The verifier checks the certificate against a CRL.
I'm sure Apple would love that too. More seriously, would that also mean all editing tools would need to re-sign a photo that was previously signed by the original sensor. How do we distinguish an edit that's misleading vs just changing levels? It's an interesting area for sure, but this inverse approach seems much trickier.
CAI’s Content Credential standard accommodates what you suggest, as far as re-signing/provenance, with a chain kind of approach. It supports embedding “ingredient thumbnails” in an image’s manifest, and/or the image’s manifest can embed or link back to source images that are in turn also signed [2].
It feels like the approach assumes a media environment where a professional wants to provably “show their work,” where authenticity adds value to a skeptical audience.
In that spirit, then, I understand CAI’s intention [0] to be to vest that judgment with the creator, and ultimately the viewer: if my purpose is to prove myself, I’d want to show enough links in the chain that the viewer checking my work can say “oh I see how A relates to B, to C,” and so on. If I don’t want to prove myself, well… then I won’t.
I don’t know Adobe’s implementation well enough to know how often they save a CC manifest, and their beta is vague in just referring to “editing history.” [1] I get the impression that they’re still dialing in the right level of detail to capture by default. Maybe even just “came from Firefly” and “Photoshop wuz here.”
But if I want to prove this Nikon Z9 recorded these pixels at this time and place, or “I am the BBC and yes I published this,” or “only the flying monkey was GenAI, the rest was real” I could conceivably put together a toolchain (independently of Adobe) to prove it in more detail.
[0] https://spec.c2pa.org/specifications/specifications/2.2/spec...
[1] https://opensource.contentauthenticity.org/docs/manifest/und...
[2] https://opensource.contentauthenticity.org/docs/c2patool/doc...
You'd have to provide both images, and let the end user determine whether they think it's misleading.
Some cameras support this, but usually only for raw.
Note that your cell phone camera is using gen AI techniques to counteract sensor noise.
Was that famous person in the background really there, or a hallucination filling in static?
Who knows at this point? So, the signatures you proposed need to have some nuance around what they’re asserting.
To be fair, I think just signing details about the way an image was assembled makes sense. Deciding on fake vs real doesn't have to be done at time of capture. We store things like the aperture size, sensitivity, camera name/model, etc in the EXIF data, including details about the image processing pipeline seems like a logical step. (With a signature verification scheme... and I guess also trying to embed that in the actual bitmap data)
There is no original image to recover, since we can't capture and describe every photon, so it's not a "fake vs real" image signature... that would be a UI choice the image viewer client would make based on the pipeline data in the image.
Years ago, I worked at Apple at the same time as Ian Goodfellow. This was before ChatGPT (I'd say around 2019).
I had the chance to chat with him, and what I remember most was his concern that GANs would eventually be able to generate images indistinguishable from reality, and that this would create a misinformation problem. He argued for exactly what you’re mentioning: chips that embed cryptographic proof that a photo was captured by a camera and haven't been modified.
[dead]