Comment by thecupisblue
3 years ago
Now, imagine this future:
You got a friend, spouse or someone close that has hundreds of pictures of you on their phone. Their phone has a "AI chip" that is used to finetune the recognition models and photo models with your AI library. Like Google Photos tags images of people you know, so does the model. It also helps sharpen images - you moved your head in an image and it was a bit blurry, but the model just fixed it, because like the original model had for the moon, it has hundreds of pictures of you to compensate.
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Good point.
For the long time digital cameras embedded in EXIF metadata about conditions on which the photo was made. Like camera model, focal length, exposure time etc
Nowadays this metadata should be extended with description of AI postprocessing operations.
> Nowadays this metadata should be extended with description of AI postprocessing operations.
Of course. But to ensure that's valid for multiple purposes we need a secure boot chain, and the infrastructure for it.
To get there we need an AI arms race. People trying to detect AI art with machine learning vs. increasing AI sophistication. Companies trying to discourage AI leaks of company secrets and reduce liability (and reduce the tragic cost of mistakes of course) vs. employees being human.
Or we could have built a responsible and reasonable government that can debate and implement that.
Maybe I'm naive. I'll take responsibility for that.
In the meantime, it's playtime for the AIs. Bring your fucking poo bags, theyre shitting everywhere (1), pack it in, pack it out.
(1) what the world didnt know, was that this was beautiful too.
> Of course. But to ensure that's valid for multiple purposes we need a secure boot chain, and the infrastructure for it. > > To get there we need an AI arms race. People trying to detect AI art with machine learning vs. increasing AI sophistication.
Or we can just recognize the lunacy of it and opt out of caring. You can't stop the flood, so you just learn to live with it. With the right view, the flood becomes unimportant.
Secure boot in practice always become slave boot. The user loses control to even control the operating system running on his device. It is the final nail in the coffin for the already dying concept of general purpose computing.
What measures can the government implement to combat this? AI image modification is realistically possible even on consumer hardware running locally. There is no going back.
Another option could be to always include the original unprocessed picture in every photo file.
Some image formats (e.g. HEIF) already allow to store multiple images in the same file.
Or just simply embed the original data from the sensor
The far greater concern is far more mundane.
Photos taken by cell phone cameras increasingly can't be trusted as evidence of the state of something. Let's say you take a picture of a car that just hit a pedestrian and is driving away.
Pre-AI, your picture might be a bit blurry, but say, it's discernible that one of the headlights had a chunk taken out of it; it's only a few pixels, but there's obviously some damage, like a hole from a rock or a pellet gun. Police find a suspect, see the car, note damage to the headlight that looks very close, get a warrant for records from the suspect, find incriminating texts or whatnot, and boom, person goes to jail for killing someone (assuming this isn't the US, where people almost never go to jail for assault, manslaughter, or homicide with a car) because the judge or jury are shown photos from the scene, taken by detectives in the street of the person's driveway, and then from evidence techs nice and close-up.
Post-"AI" bullshit, the AI sees what looks like a car headlight, assumes the few-pixels damage is dust on the sensor/lens or noise, and "fixes" the image, removing it and turning it into a perfect-looking headlight.
Or, how about the inverse? A defense attorney can now argue that a cell phone camera photo can't be relied upon as evidence because of all the manipulation that goes on. That backpack in a photo someone takes as a mugger runs away? Maybe the phone's algorithm thought a glint of light was a logo and extrapolated it into the shape of a popular athletic brand's logo.
I’d just like them to fix the problem where license plates are completely unreadable by most consumer cameras at night. It’s almost as though they are intentionally bad. (The plate ends up as a blown out white rectangle.)
The recent kyle rittenhouse trial had an element that hinged on whether apple's current image upscaling algorithm uses AI, and hence whether what you could see in the picture was at all reliable. The court system is already aware of and capable of dealing with these eventualities.
“Aware of” does not necessarily mean “capable of dealing with”. Forensics is generally bad science, yet gets admitted into court all the time. This occurs despite many legal textbooks, papers, and court opinions highlighting the deficiencies.
The question was more general, if the iPad zooming introduced any different pixels (e.g. a purple pixel between red and blue). Or, "uncharged pickles" as the judge put it.
It doesn't even need AI to be problematic. Pinch-zoom has no business being used in the courtroom as it inherently can introduce issues. However, a fixed integer ratio blowup of the image shouldn't be problematic. (2:1 is fine. 1.9:1 inherently can't guarantee it doesn't introduce artifacts.)
Well it's not capable of dealing with it because they found apple's zoom was unreliable and it contributed to the guy getting off
The guy got off because he was legitimately acting in self-defense.
I thought it was really funny in the 1980s that people in medical imaging were really afraid to introduce image compression like JPEG because the artifacts might affect the interpretation of images but today I see article after article about neural image enhancement and it seems almost no concern that a system like that would be great at hallucinating both normal tissue and tumors.
So far as law and justice goes it is the other way around too. If it is known to be possible that cameras can hallucinate your identity, it won't be possible to use photographic proof to hold people to account.
It seems fairly easy to bake a chain of custody into your images. Sensor outputs a signed raw image, AI outputs a different signed “touched up” image. We can afford to keep both in this hypothetical future; use whichever one you want.
Once generative AI really takes off we will need some system for unambiguously proving where an image/video came from; the solution is quite obvious in this case and many have sketched it already.
The images generated by SLR and mirrorless cameras are already signed with device embedded keys during EXIF embedding. Every manufacturer sells such verification systems to law enforcement or other institutions to verify such images.
Sometimes there are exploits which extract these keys from the cameras themselves, but I don't hear them nowadays.
One of the older products: https://imaging.nikon.com/lineup/software/img_auth/index.htm
Until some PM says "why do we have both images, our data shows that 99.5% of users don't use the raw image, let's remove that feature."
And the answer is “spam filters and AI personal curation agents will drop any image without chain of custody from every feed that claims to be about reality”.
In a world where any image or video can be generated, chain of custody to a real-world ground-truth will be vitally important.
4 replies →
Simple solution: keep the raw image and AI it on the fly (this is a hypothetical future remember)
Bonus feature: new display devices get better trained / new AI features for free
2 replies →
In a functional government, that data is mandated And regulated
Then someone takes a photo of their TV screen. Presto, instant chain of custody for any image you want!
It’s a fair point but with high enough resolution (and perhaps GPS baked into the trusted data) I suspect it would be very hard to actually forge a digital image from an analog source.
Likewise depth fields and other potential forms of sensor augmentation.
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." [1]
If any of you young folks haven't watched Ghost in the Shell, just close this tab and do that.
[1] https://ghostintheshell.fandom.com/wiki/Laughing_Man
`Ghost in the Shell: Stand Alone Complex`
If possible I suggest the original version, though the CGI in the retouched anime wasn’t as bad as I expected.
I feel like with ScarJo lately you either get "big budget movie where she's phoning it in" or "small movie most people won't watch where she's a great actress. Does this fall into either of those categories?
I was thinking of the 1995 animated film. I haven't seen the 2017 ScarJo version, even though she is very pretty.
2 replies →
It's decent!
[dead]
Obviously someone who has good enough position to take semi-clear photo and who knows you so well, that has phone full of your face, will not recognize you directly, but will be convinced that you are robber after looking at photo. At this point we can go full HN and assume that you will be convinced anyway, because judge is GPT-based bot.
This "future" is present in current Pixel lineup btw. Photos are tagged as unblured, so for now you can still safely take a selfie with your friends.
They should call the chip XeroxAI.
https://news.ycombinator.com/item?id=29223815
imagine you want to "scan" a document using camera app like many people do, and ai sees blurry numbers and fixes then for you. when will you notice that some numbers even that look clear are different than on original document?
Been there: https://www.nbcnews.com/technolog/copier-conundrum-xerox-mac...
Would you be surprised to learn that this has been a thing for years already?
https://news.ycombinator.com/item?id=6156238
I think this is so we'll known case that Samsung wouldn't make such trivial mistake and if ai detects you are photographing a document it'll disable ai magic automatically, but imagine scenario eg. where you are trying to photograph some phone number from some banner, would ai also guess it shouldn't mess with numbers there? imagine someone with poor eyesight wants to read this way something from a sign that is far away and I would bet that super zoom would be attractive feature for person with such sight issues
AI-based image generation is surely already good enough that a single digital photo can't count as evidence alone. But your scenario doesn't make much sense to me - are you suggesting AI will have reached a point it's stored and trained on images of almost everyone's faces, to the point it could accurately/undetectably substitute a blurry face with the detailed version of an actual individual's face it happens to think is similar? I'd be far more worried about deliberate attempts to construct fake evidence - it seems inevitable that eventually we'll have technology to cheaply construct high-quality video and audio media that by current standards of evidence could incriminate almost anyone the framer wanted to.
Look similar to a celebrity? Your face gets replaced, because the number of photos of the celebrity in the corpus outweighs photos of you. And when those doctored photos end up in the corpus, weighting will be even further towards the celebrity So people who look less like the celebrity get replaced, because it is almost certainly them according to the AI. Feeding back until everyone gets replaced by a celebrity face. And then the popular celebrities faces start replacing the less well known celebrities. And we end up with true anonymity, with everyone's face being replaced by John Malkovich.
> Congratulations, you are now a robber.
Yeah, but in the future the government will know your precise location, all day, every day, so at least you'll have an alibi.
However, their omnipresent surveillance data will show that eight years, seven months, and thirteen days earlier you cut off the DA's third cousin while driving on the freeway, so the DA will conveniently forget to present this alibi as evidence.
AI isn't the thing to be worried about. People with power abusing AI is the thing to be worried about.
Whether zooming in on an image on iPad adds "extra" details was already a contentious discussion during Kyle Rittenhouse trial. The judge ultimately threw that particular piece of evidence out, as the prosecution could not prove that zooming in does not alter the image.
Now imagine that we can use a future full homomorphic encryption and train models without revealing our private data.
Obligatory throwback to the Xerox incident: https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Sounds like pretty standard forensic science, like bite marks and fingerprints.