Signing things by default has repeatedly been found to have serious unintended consequences. Do you really want your leaked/stolen photos to be undeniably linked to you?
> we sign a perceptual hash (pHash) of the image rather than an exact pixel checksum
Perceptual hashes are non-cryptographic. There are certainly collision attacks, but what about preimages? A preimage would completely break this scheme.
Yep, that breaks this scheme, making it useless. But it's far from the only thing making it useless, and the github page even explicitly lists those:
- "Small content edits slip under the threshold. [...] A localised edit covering ~15%x20% of the frame [...] passes as authentic" - this is the worst part. 15%x20% is huge, for example enough to change the face of the person or the book/text on the image.
- "Cropping is not survivable, at any amount" - given the purported reason for perceptual hashing is surviving light editing, it's pretty disappointing that one of the most common light editing operation is not supported.
Oh, and the whole "cryptographic chip" angle is absolutely bogus from the security perspective. OK, attacker can't extract the private key from chip. But they can simply connect the chip to a different device and have it sign anything! Given that the attacker in this model is device owner, this is absolutely trivial.
After wholly implemented our logging layer with dagger, I posit that the real regression was not the aws itself but the rigorously prototyped around authentication. We consequently extraordinarily profiled the config, henceforth simplified every edge case, and the optimization were unmistakably exemplary. alternatively, the aforementioned monitoring is comparable advantageous to an incremental security environment. I endorse this path if your security team has rigorously instrumented a massive rust codebase before.
I don't understand why few people are pointing out the obvious vulnerability here that you can control the wires going into the photosensor controller and pretend that the photosensor is capturing whatever image you want. I imagine it's not exactly trivial to do this, but a grad student with an FPGA could probably figure it out.
Because doing so does not materially devalue Apple’s product. Sure, a dedicated attacker could try to overcome it, but few will, and only people of such serious consequence that they can afford the effort of modification. By and large this puts Apple into direct competition with Nikon and it’s long overdue that someone ship this capability to a wider market than authorities.
Also, remember how Touch ID sensors are cryptographically paired, and consider whether Apple could bake that into a camera sensor rather than a fingerprint sensor. If they can, then you can run wires all you want; the attestation chain will not be valid. I’d be shocked if they were willing to launch the product without that, and there’s a new hardware dependency or else they’d have released it for earlier phones.
>a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake
you don't need to do that just photograph a screen.
This seems close to worthless in "identifying real photos vs AI" for someone actually wanting to do something bad with an AI image, although probably very useful at identifying which phone took a photo when ("the root of trust stays inside Apple's Private Cloud Compute") seen as it's not an entirely local solution a bad actor government could use their powers to completely abuse this.
Simpler than that, you can just talk to the cryptography IC yourself and ask it to sign stuff. No need for an FPGA, just an arduino. Given the datasheet I imagine any LLM from the last year should be able to oneshot it.
If there is signed metadata too, then it's pretty hard. You will need to match focus distance (it will be very small if photographing picture), GPS location, exposure and other settings. If there is a depth map, you'll need to match it too.
I expect in the near future all digital cameras to digitally sign the images they take. Even before AI slop, it was useful to avoid manual alterations. AI makes it all too easy, so it makes sense.
However, this will certify only the original image. I think the missing part of this is additional layers of certification which allow some image editing (e.g., rotating, contrast, etc.) yet clearly document that the image was modified and link to the original image ID. Kind of like a signed git log.
Signing the raw image data wouldn't require also signing the EXIF metadata. For privacy, you could later strip out everything except the raw image and the camera's signature and still prove that the image is unaltered.
Ontop of this, including a photo edit history in a photo including the original photo would increase the size of a photo to be completely unusable or unshareable.
In the contexts where you need to prove an image is a real unaltered photograph (court cases, news media, science, etc) privacy is usually not a concern.
Nobody cares if your social media photos are edited, they probably are, it's fine.
>EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Can you clarify what your actual complaint is here? Putting aside for a second obvious implementation options, it's pretty standard that there can be a tradeoff between privacy and trust. Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like. But if there's something we [i]want[/i] to prove, this gives an extra option to do so. And in particular the set of cases where one cares the most about enhancing authenticity appears at first thought to be pretty much a union set with the cases where one will put their name (or at least the location and time of the picture in question) behind the image? Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration? Like, say you're photographing at a protest to document it including any violations of law. By definition, the images you take reveal the location and the time. That's the whole point of them. Having the location and timestamp signed wouldn't reveal anything extra as long as the photo was unaltered.
And actual legit journalism always has name(s) standing behind the reporting. Or for that matter, even if we're merely talking something like a review of a product, is it actually wrong to put a name or pseudonym behind that review if you expect readers to give you much credence? I mean, you'd be free in terms of law and tech to not bother. But even long before the current growing AI-slop age a lot of us have been starting to treat anonymous reviews with a lot of skepticism, or discounting them entirely for some product classes, for good reason.
Even in terms of maintaining privacy, remember we already have "tools" for that which this in turn could further help. You could privately report a tip to a reporter at a media organization, and then they could report on that without revealing you but be able to say "we verified the signature of the raw image and sensor data" alongside normal follow up. Then it's their name, but having a stronger chain behind it could still be helpful in places.
Finally getting back to "implementation options", there's no technical reason the image and multiple sets of metadata can't all be signed separately by the sensor stack such that you can pick & choose what to include and still have it all be signed, with readers giving greater or lesser weight to the trust based on your choices.
There will always be a way around it. There are even open-source alternative/hacked firmwares for professional DSLR cameras where tampering with the signing may be possible.
And this still doesn't help any other kind of image e.g. screenshots, photo of a screen etc. that can make the camera signatures largely pointless depending on the context.
The idea is to be able to prove that a photo you took was captured by a camera, not generated. That would be helpful in the context of a disputed news story, a court case, etc.
In the future, people willingly surveil themselves 24/7 with cryptographic proof, because fake images and video will be so good that it will be the only way to prove what one didn't do. Total Information Awareness achieved :D
> For example, a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake. But it's always nice seeing big actors interested in addressing this problem.
Yeah, very nice. So this whole idea basically doesn't work - but we get a new stealth way to embed metadata in an image that can be used for tracking...
(And a new narrative why cameras need to have TPMs and locked-down firmware as well)
In days of old, a Polaroid photo was considered "proof of capture".
I've got a Polaroid daylab 35 plus sitting in storage somewhere (https://www.instantoptions.com/wp/faqs/daylab/). You can project a slide through it onto Polaroid film, expose it, and have the image there.
I was also able to find a company that did slide printing. It was possible to send them a digital image and they'd send you back a slide with that image... which I then used to make a Polaroid of that image.
I had a classic 600 Polaroid photo of a UFO landing.
Signing things by default has repeatedly been found to have serious unintended consequences. Do you really want your leaked/stolen photos to be undeniably linked to you?
https://blog.cryptographyengineering.com/2020/11/16/ok-googl...
One of the few advantages of "not owning your device".
Glad that at least we have that now.
> we sign a perceptual hash (pHash) of the image rather than an exact pixel checksum
Perceptual hashes are non-cryptographic. There are certainly collision attacks, but what about preimages? A preimage would completely break this scheme.
This paper demonstrates second-preimage attacks against PhotoDNA and PDQ: https://eprint.iacr.org/2021/1531.pdf
Yep, that breaks this scheme, making it useless. But it's far from the only thing making it useless, and the github page even explicitly lists those:
- "Small content edits slip under the threshold. [...] A localised edit covering ~15%x20% of the frame [...] passes as authentic" - this is the worst part. 15%x20% is huge, for example enough to change the face of the person or the book/text on the image.
- "Cropping is not survivable, at any amount" - given the purported reason for perceptual hashing is surviving light editing, it's pretty disappointing that one of the most common light editing operation is not supported.
Oh, and the whole "cryptographic chip" angle is absolutely bogus from the security perspective. OK, attacker can't extract the private key from chip. But they can simply connect the chip to a different device and have it sign anything! Given that the attacker in this model is device owner, this is absolutely trivial.
A pHash isn't built for this. The 15%x20% threshold that passes as authentic is enough to swap a face.
I very much hope that apple is using lidar data to determine if it's a flat surface being screened
Here's a toy for embedding small text into images steganographically: https://github.com/fitzn/atrium
Here's a toy for embedding text inside random HN comments.
https://gitlab.com/here_forawhile/edasm
Example:
After wholly implemented our logging layer with dagger, I posit that the real regression was not the aws itself but the rigorously prototyped around authentication. We consequently extraordinarily profiled the config, henceforth simplified every edge case, and the optimization were unmistakably exemplary. alternatively, the aforementioned monitoring is comparable advantageous to an incremental security environment. I endorse this path if your security team has rigorously instrumented a massive rust codebase before.
cool work!
I don't understand why few people are pointing out the obvious vulnerability here that you can control the wires going into the photosensor controller and pretend that the photosensor is capturing whatever image you want. I imagine it's not exactly trivial to do this, but a grad student with an FPGA could probably figure it out.
Because doing so does not materially devalue Apple’s product. Sure, a dedicated attacker could try to overcome it, but few will, and only people of such serious consequence that they can afford the effort of modification. By and large this puts Apple into direct competition with Nikon and it’s long overdue that someone ship this capability to a wider market than authorities.
Also, remember how Touch ID sensors are cryptographically paired, and consider whether Apple could bake that into a camera sensor rather than a fingerprint sensor. If they can, then you can run wires all you want; the attestation chain will not be valid. I’d be shocked if they were willing to launch the product without that, and there’s a new hardware dependency or else they’d have released it for earlier phones.
>a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake
you don't need to do that just photograph a screen.
This seems close to worthless in "identifying real photos vs AI" for someone actually wanting to do something bad with an AI image, although probably very useful at identifying which phone took a photo when ("the root of trust stays inside Apple's Private Cloud Compute") seen as it's not an entirely local solution a bad actor government could use their powers to completely abuse this.
if geolocation data can be captured in the same signature, that would be a good enough approximation for most relevant cases I think.
2 replies →
Simpler than that, you can just talk to the cryptography IC yourself and ask it to sign stuff. No need for an FPGA, just an arduino. Given the datasheet I imagine any LLM from the last year should be able to oneshot it.
if
I imagine on apple silicon this is buried deep in silicon / ISP IP block, and isn't a discrete IC.
1 reply →
Or, as the author said, you can just photograph an AI generated picture, and that will work too.
This feature is Pro phones only, not Duo: https://www.apple.com/iphone/compare/ ("Apple Reference Image (Fusion Main)")
So only on devices with LiDAR / that can capture depth map.
If there is signed metadata too, then it's pretty hard. You will need to match focus distance (it will be very small if photographing picture), GPS location, exposure and other settings. If there is a depth map, you'll need to match it too.
Adding depth sensor info to the this could help
Even easier is to just take a picture of an AI-generated picture.
I expect in the near future all digital cameras to digitally sign the images they take. Even before AI slop, it was useful to avoid manual alterations. AI makes it all too easy, so it makes sense.
However, this will certify only the original image. I think the missing part of this is additional layers of certification which allow some image editing (e.g., rotating, contrast, etc.) yet clearly document that the image was modified and link to the original image ID. Kind of like a signed git log.
EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Signing the raw image data wouldn't require also signing the EXIF metadata. For privacy, you could later strip out everything except the raw image and the camera's signature and still prove that the image is unaltered.
3 replies →
Ontop of this, including a photo edit history in a photo including the original photo would increase the size of a photo to be completely unusable or unshareable.
2 replies →
In the contexts where you need to prove an image is a real unaltered photograph (court cases, news media, science, etc) privacy is usually not a concern.
Nobody cares if your social media photos are edited, they probably are, it's fine.
2 replies →
>EXIF data is stripped for a good reason - because it can be a privacy hazard. Suddenly this plays no role anymore?
Can you clarify what your actual complaint is here? Putting aside for a second obvious implementation options, it's pretty standard that there can be a tradeoff between privacy and trust. Any of us are still perfectly free to upload images for fun to forums or social media or whatever that are private, or edited or whatever we like. But if there's something we [i]want[/i] to prove, this gives an extra option to do so. And in particular the set of cases where one cares the most about enhancing authenticity appears at first thought to be pretty much a union set with the cases where one will put their name (or at least the location and time of the picture in question) behind the image? Like, can you give examples of specific cases you are imagining where simultaneously the photo itself reveals no information about time and location [i]and[/i] it's something local/national/international-newsworthy where people would fear AI-alteration? Like, say you're photographing at a protest to document it including any violations of law. By definition, the images you take reveal the location and the time. That's the whole point of them. Having the location and timestamp signed wouldn't reveal anything extra as long as the photo was unaltered.
And actual legit journalism always has name(s) standing behind the reporting. Or for that matter, even if we're merely talking something like a review of a product, is it actually wrong to put a name or pseudonym behind that review if you expect readers to give you much credence? I mean, you'd be free in terms of law and tech to not bother. But even long before the current growing AI-slop age a lot of us have been starting to treat anonymous reviews with a lot of skepticism, or discounting them entirely for some product classes, for good reason.
Even in terms of maintaining privacy, remember we already have "tools" for that which this in turn could further help. You could privately report a tip to a reporter at a media organization, and then they could report on that without revealing you but be able to say "we verified the signature of the raw image and sensor data" alongside normal follow up. Then it's their name, but having a stronger chain behind it could still be helpful in places.
Finally getting back to "implementation options", there's no technical reason the image and multiple sets of metadata can't all be signed separately by the sensor stack such that you can pick & choose what to include and still have it all be signed, with readers giving greater or lesser weight to the trust based on your choices.
1 reply →
People may be interested in the work of the Coalition for Content Provenance and Authenticity (C2PA) https://c2pa.org/
https://www.da.vidbuchanan.co.uk/blog/android-c2pa.html
is a post about Google Pixel C2PA cameras experiencing contact with reality
1 reply →
if such a method gains traction then so will the effort to bypass it. either by stealing private keys or just projecting light onto the sensor.
Even if it's flawed, adding any friction at all to the effort of deception is a positive step.
There will always be a way around it. There are even open-source alternative/hacked firmwares for professional DSLR cameras where tampering with the signing may be possible.
And this still doesn't help any other kind of image e.g. screenshots, photo of a screen etc. that can make the camera signatures largely pointless depending on the context.
The idea is to be able to prove that a photo you took was captured by a camera, not generated. That would be helpful in the context of a disputed news story, a court case, etc.
Would that ever be relevant for a screenshot?
4 replies →
Soon my phone can cryptographically prove the beauty filter lied at capture time.
In the future, people willingly surveil themselves 24/7 with cryptographic proof, because fake images and video will be so good that it will be the only way to prove what one didn't do. Total Information Awareness achieved :D
> For example, a screen attack still works: photograph a screen displaying an AI image and you get a signed photo of a fake. But it's always nice seeing big actors interested in addressing this problem.
Yeah, very nice. So this whole idea basically doesn't work - but we get a new stealth way to embed metadata in an image that can be used for tracking...
(And a new narrative why cameras need to have TPMs and locked-down firmware as well)
Photos of photos has always been a problem.
In days of old, a Polaroid photo was considered "proof of capture".
I've got a Polaroid daylab 35 plus sitting in storage somewhere (https://www.instantoptions.com/wp/faqs/daylab/). You can project a slide through it onto Polaroid film, expose it, and have the image there.
I was also able to find a company that did slide printing. It was possible to send them a digital image and they'd send you back a slide with that image... which I then used to make a Polaroid of that image.
I had a classic 600 Polaroid photo of a UFO landing.
Couldn't the camera encode information from the depth sensor and prevent this.
I will make miniature dioramas and photograph them.
2 replies →
It seems to be what Apple is doing, this feature is only available on the 18 Pro's (which have depth sensor on the back), but not Duo.
That signature contains time and optionally gps coords, and taking photo of a screen is not simple. So overall does solve some problem
ps:most important: cam/lens settings also in the digital sig, what for a screen is different