Comment by Jensson

This exact situation came up during the Kyle Rittenhouse trial. The prosecution wanted to present an image of Rittenhouse. It was materially important whether or not the picture depicted Rittenhouse with a raised gun; I looked myself and couldn’t tell because it was a dark picture from a distance (if I remember correctly, it was a still from a drone video). Since the relevant part of the image was so small, the prosecution was going to zoom the photo in on an iPad, but the defense objected on the basis that the iPad zoom algorithms could be inserting detail not present in the original image.

This seemed a plausible enough objection to me. Although a fairly techy guy, I was (and am) not familiar with the specifics of Apple image processing, but at the time I had a vague awareness that Apple had been heavily advertising its use of AI algorithms to improve the quality of images. Whether that affects zooms specifically I don’t know—but it’s not an outlandish question.

The judge did an eminently reasonable thing: he disallowed the zoomed evidence on its own, but allowed it to be re‐entered if the prosecution provided an expert witness to testify that zooming the photo didn’t meaningfully change it. For this, he was pilloried by the tech media.

Take Ars Technica, for example: they used the headline “You shall not pinch to zoom: Rittenhouse trial judge disallows basic iPad feature,” and prominently displayed the judge’s words “I know less than anyone [about technology],” as if the right thing for a judge who knows nothing about technology to do would be to determine the merits of technical evidence on his own rather than ask for an expert witness. It’s not like it would have been hard for the prosecution to find one. https://arstechnica.com/tech-policy/2021/11/rittenhouse-tria...

Anyone who’s experimented with even non‐AI‐based upscalers knows that changing the algorithm can connect or disconnect catty‐cornered objects, introduce curves, and so on. I was shocked (well, not that shocked, given the heavily partisan interest in the case) that the tech media was so confident zooming an image couldn’t possibly meaningfully change it.

In the end, the image was displayed on a big‐screen TV (which probably used some other upscaling algorithm like bilinear, not that anyone in court was technical enough to point it out). The prosecution asked whether/asserted that Rittenhouse had raised his gun in the image, and Rittenhouse said it was not raised. So the exact details of how the small few pixels in the image got upscaled turned out to be pivotal in the end after all.

And it is possible that resulting image is a better match to reality than the raw data coming out of the camera .

iPhones, no, there's no AI replacement or synthesis of objects from the camera. There were Android phones doing this (famously I think it was Samsung where it would replace images of the moon with a different image of the moon), and the Photos app has AI manipulation features. And most of the time, Apple's noise removal algorithm actually removes detail from images, most notably making text and straight lines wobbly.