Sensor quality in phones goes down, AI makes up for it because good sensors are expensive, but compute time in the cloud on Samsung owned servers is cheap. You take a picture on a crappy camera, and Samsung uses AI to "fix" everything. It knows what stop signs, roadways, busses, cars, stop lights, and more should look like, and so it just uses AI to replace all the textures.
Samsung sells what's on the image to advertisers and more with the hallucinated data. People can't tell the difference and don't know. They "just want a good looking picture". People further use AI to alter images for virtual likes on Tiktok and Insta.
This faked data, submitted by users as "real pics in real places" is further used to train AI models that all seem to think objects further away have greater detail, clarity, and cleanliness than they should.
You look at a picture of a park you took, years before, and could have sworn the flowers were more pink, and not as red. You are assured, by your friend who knows it all, that people's memories are fallible; hallucinating details, colors, objects, sizes, and more. The image, your friend assures you further? "Advanced tech captured its pure form perfectly".
And thus, everyone will demand more clarity, precision, details, and color where their eyes don't remember seeing.
You got a friend, spouse or someone close that has hundreds of pictures of you on their phone. Their phone has a "AI chip" that is used to finetune the recognition models and photo models with your AI library. Like Google Photos tags images of people you know, so does the model. It also helps sharpen images - you moved your head in an image and it was a bit blurry, but the model just fixed it, because like the original model had for the moon, it has hundreds of pictures of you to compensate.
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
For the long time digital cameras embedded in EXIF metadata about conditions on which the photo was made. Like camera model, focal length, exposure time etc
Nowadays this metadata should be extended with description of AI postprocessing operations.
Photos taken by cell phone cameras increasingly can't be trusted as evidence of the state of something. Let's say you take a picture of a car that just hit a pedestrian and is driving away.
Pre-AI, your picture might be a bit blurry, but say, it's discernible that one of the headlights had a chunk taken out of it; it's only a few pixels, but there's obviously some damage, like a hole from a rock or a pellet gun. Police find a suspect, see the car, note damage to the headlight that looks very close, get a warrant for records from the suspect, find incriminating texts or whatnot, and boom, person goes to jail for killing someone (assuming this isn't the US, where people almost never go to jail for assault, manslaughter, or homicide with a car) because the judge or jury are shown photos from the scene, taken by detectives in the street of the person's driveway, and then from evidence techs nice and close-up.
Post-"AI" bullshit, the AI sees what looks like a car headlight, assumes the few-pixels damage is dust on the sensor/lens or noise, and "fixes" the image, removing it and turning it into a perfect-looking headlight.
Or, how about the inverse? A defense attorney can now argue that a cell phone camera photo can't be relied upon as evidence because of all the manipulation that goes on. That backpack in a photo someone takes as a mugger runs away? Maybe the phone's algorithm thought a glint of light was a logo and extrapolated it into the shape of a popular athletic brand's logo.
The recent kyle rittenhouse trial had an element that hinged on whether apple's current image upscaling algorithm uses AI, and hence whether what you could see in the picture was at all reliable. The court system is already aware of and capable of dealing with these eventualities.
I thought it was really funny in the 1980s that people in medical imaging were really afraid to introduce image compression like JPEG because the artifacts might affect the interpretation of images but today I see article after article about neural image enhancement and it seems almost no concern that a system like that would be great at hallucinating both normal tissue and tumors.
So far as law and justice goes it is the other way around too. If it is known to be possible that cameras can hallucinate your identity, it won't be possible to use photographic proof to hold people to account.
It seems fairly easy to bake a chain of custody into your images. Sensor outputs a signed raw image, AI outputs a different signed “touched up” image. We can afford to keep both in this hypothetical future; use whichever one you want.
Once generative AI really takes off we will need some system for unambiguously proving where an image/video came from; the solution is quite obvious in this case and many have sketched it already.
Obviously someone who has good enough position to take semi-clear photo and who knows you so well, that has phone full of your face, will not recognize you directly, but will be convinced that you are robber after looking at photo. At this point we can go full HN and assume that you will be convinced anyway, because judge is GPT-based bot.
This "future" is present in current Pixel lineup btw. Photos are tagged as unblured, so for now you can still safely take a selfie with your friends.
imagine you want to "scan" a document using camera app like many people do, and ai sees blurry numbers and fixes then for you.
when will you notice that some numbers even that look clear are different than on original document?
AI-based image generation is surely already good enough that a single digital photo can't count as evidence alone. But your scenario doesn't make much sense to me - are you suggesting AI will have reached a point it's stored and trained on images of almost everyone's faces, to the point it could accurately/undetectably substitute a blurry face with the detailed version of an actual individual's face it happens to think is similar?
I'd be far more worried about deliberate attempts to construct fake evidence - it seems inevitable that eventually we'll have technology to cheaply construct high-quality video and audio media that by current standards of evidence could incriminate almost anyone the framer wanted to.
Whether zooming in on an image on iPad adds "extra" details was already a contentious discussion during Kyle Rittenhouse trial. The judge ultimately threw that particular piece of evidence out, as the prosecution could not prove that zooming in does not alter the image.
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Sounds like pretty standard forensic science, like bite marks and fingerprints.
Basically this.. As "neat" as AI "improvement" is, I don't think it has any actual value, I can't come up with any use-case where I can accept it. "Make pictures look good by just hallucinating stuff" is one of the harder ones to explain, but you did it well..
Another thing, pictures for proof and documentation, maybe not when they're taken but after the fact, for historical reasons, or forensics.. We can't have every picture automatically compromised as soon as it's taken. (Yes, I know that photoshop is a thing, but that's a very deliberate action, which I believe it should be)
I think the main use case is "I'm a crummy photographer and all I want is something to remind me that I was there" and "Look at my cat. Look! Look at her!"
That's me. I'm a lousy photographer, as evidenced by all of the photos I shot back when film actually recorded what you pointed it at. My photography has been vastly improved by AI. It hasn't yet reached the point of "No, you idiot, don't take a picture of that. Go left. Left! Ya know what, I'm just gonna make something up," but it should.
I imagine there will remain a use case for people who can actually compose good shots. For the remaining 99% of us, we'll use "Send the camera on vacation and stay home; it's cheaper and produces better pictures" mode.
Interestingly enough one of the reason Sonys flagships perform really badly in comparisons is because they are weak at computational photography. So even when the sensor is great it looks too real, which people don't like.
How about using AI for sensor fusion when you have images from multiple different kinds of lenses (like most smartphones today)? I was under the impression this was the main reason why AI techniques became popular in smartphone cameras to begin with
Good for situations where you aren’t expecting or care about realism in this detail. AI hallucinations will be amazing for entertainment, especially games.
I don't already fully trust the images, audio and videos I take with the phone.
I'm working close to HW and I actively use the camera/picture and videos for future reference and debugging. It's small, fits in your pocket, and the bloody thing can record at 240fps to booth!
Until you realize there's so much post-processing done on the images, video and audio you can't really trust and can't really know if you can turn it all off. The reality is that if you could, you'd realize there's no free lunch. It's a small sensor, and while we had huge improvements in sensor and small lenses, it's still a small sensor.
Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
In the end, they're consumer devices. "Does it look good -> yes" is what thrums everything in this market. Expect the worst.
> Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
This has been true of consumer digital cameras for 25 years. It's not new to or exclusive to smartphone cameras. It's not even exclusive to consumer cameras as professional ones costing many times more also do a bunch of image processing before anything is committed to disk.
I don’t know about android, but at least with my iPhone I’m pretty sure there are apps that can capture raw sensor data. Additionally I do have the ability capture Apple ProRAW format at of the photos. I don’t actually know if these images are still processed though.
"You just took a picture of the Eiffel Tower. We searched our database and found 2.4 million public pictures taken from the same location and time of day. Here are 30,000 photos that are identical to yours, except better. Would you like to delete yours and use one of them instead?"
There was a pretty neat Google project a few years back that showed time-lapse videos of buildings under construction created entirely through publicly posted images that people had happened to take at the same spot over time.
I wonder if that'll ever cause legal problems in the future. Sorry, that photo someone took where the accused was in background at a party some years ago? He was kinda blurry and those facial features have been enhanced with AI, that evidence will have to be thrown out. Or maybe the photo is of you, and you need it as an alibi..
This is actually exactly what happened during the Kyle Rittenhouse case. A lawyer for the defense tried to question video evidence because of AI being used to enhance zoomed shots.
"Good sensors are expensive"-fun-fact: Mid-range CCTV cameras often have bigger sensors (1/1.8" or 1/1.2") and much faster lenses than an iPhone 13 Pro Max (1/1.9" for the main camera). The CCTV camera package is of course far bigger though. But still kinda funny in a way.
Edit: And the lenses on these are not your granddads computar 3-8/1.0, either. Most of the CCTV footage we see just comes from old, sometimes even analog, and lowest-bidder installations.
Bruce Sterling I think had a story in that direction. A polaroid camera producer would develop photos which would've been algorithmically enhanced so that their clients consider themselves better photographers and their cameras superior. I'm regularly updated for it for the last few years when cameras are more and more their software.
Edit: fixed the author's name. Cannot find the exact story though.
This has in some ways been happening for decades. There are a few countries where the way to take a good portrait of a person is to over expose the photo, so skin tones are lighter. People bought the cameras and phones that did this by default (by accident or design in the 'portrait mode' settings). They didn't want realism.
This is just a progression of the nature of our human world - we have been replacing reality with the hyperreal for millennia, and the pace only accelerates. The map is the territory. Korzybski was right, but Baudrillard even more so.
Eventually people won’t care much for clarity and precision, that’s boring. The real problem is that everything that can be photographed will eventually have been photographed in all kinds of ways. What people really want is just pictures that look more awesome, in ways other people haven’t seen before.
So instead, raw photos will be little more than prompts that get fed to an AI that “reimagines” the image to be wildly different, using impossible or impractical perspectives, lighting, deleted crowds, anything you can imagine, even fantasy elements like massive planets in the sky or strange critters scurrying about.
And thus, cameras will be more like having your own personal painter in your pocket, painting impressions of places you visit, making them far more interesting than you remember and delighting your followers with unique content. Worlds of pure imagination.
I like the story, but I think people will notice pretty quickly as almost everyone reviews their photos right after taking them (so they can compare them with what they see in reality)
True, its just a fun story. This reddit post makes it clear, though, that while people will review the images carefully, they may still not be able to accurately determine differences.
Just take the story above with one more minor step: You snap a pic of the park, briefly glanced at it to make sure it wasn't blurry (which the AI would have fixed anyway) or had an ugly glare (it did, the AI fixed it) or worse a finger (the AI also fixed that).
You're satisfied the image was captured faithfully and you did a good job holding your plastic rectangle to capture unseen sights. You didn't look closely enough to notice all the faked details, because they were so good.
This fake moon super enhance? It already proves people will fall for it. I could easily see people not realizing AI turned the flowers in the picture more red, or the grass just a little too green, etc.
I guess I havent noticed that people do that for things other than selfies.
I generally just burst-mode-scan an area or scenery location and later that night, or when I add to Strava or wherever, I have an old school contact sheet (but with 60-80 images per thing) to look though. Then narrow it down to 5-10, pick the one or two I like best and discard the rest.
It's sorta like this already, in the _present_ - people post photos with filters all the time, smart phone cameras color-correct and sharpen everything with AI (not just Samsung's). It'll just become more and more commonplace
The problem is that this particular AI enhancement was not advertised as such. Also, in the linked article it was putting moon texture on ping pong balls, which seemed like overzealous application of AI. Samsung could have marketed it as "moon enhancement AI" or something like that, which would be more honest.
My worry about these features becoming commonplace is that if everyone just leave those features enabled, we would end up with many boring photos because they all look similar to each other. The current set of photo filters, even though they seem to be converging on particular looks, at least don't seem to invent as much detail as pasting a moon that's not there.
I still argue that my Galaxy Note 8 took cleaner pictures in general than my Galaxy Note 20. Everything feels overly processed, even in "pro" mode with all processing settings turned off.
I’ve always thought this was the final outcome of all AI; a feedback loop. Same when ChatGPT starts using things ChatGPT wrote itself as references to train itself.
We already have people demanding higher definition televisions to watch AI-sharpened 4K restorations of old films whose grain and focus would annihilate any details that small worth seeing.
There's an arms race between people adding nonexistent details to old films and people manufacturing televisions with dense enough pixels to render those microscopic fictions. Then they lay a filter over it all and everything becomes smooth gradients with perfectly sharp edges.
People want this. It's already happening. There was a post on the Stable Diffusion reddit where someone ran a picture of their grandparents through it to colorize and make it "look nicer". But it made significant changes to their clothes and hallucinated some jewelry they weren't wearing, along with some subtle changes in the faces. It's not real anymore, but hey it looks nicer right?
What you're imagining is Hyperreality from Simulacra and Simulation and has been happening since the invention of the television, and later the internet.
AI will accelerate this process exponentially and just like in The Matrix, most people will eventually prefer the simulation to reality itself.
This is my exact worry with things like chat gpt polluting the scrapable internet. The feedback loop might eventually ruin whatever value the models currently have by filling them with incorrect but plentiful generated nonsense.
I was thinking of a scenario. My children are adults and browsing photos of themselves as children. They come across a picture of the family on a vacation to the beach. They dimly remember it, but the memories are fond. They notice they are holding crisp ice cold cans of Coca Cola Classic (tm). They don’t remember that part very well. Mom and dad rarely let them drink Coke. Maybe it was a special occasion. You know what, maybe it would be fun to pick up some Coke to share with their kids!
So a future where reality and history are subtly tweaked to the specifications of those willing to pay…
Google already scans your photos folder and offers enhancements, stitches together panoramas and so on. So inserting product placement is totally believable.
These scenarios were much talked about a decade back in relation to advertising on photographs on Facebook, specially with Coca Cola and other popular brands.
That isn't far from how iphones work now. They have mediocre cameras, people only think they are good because they throw a lot of AI image enhancement at it.
Is Apple’s AI adding hallucinated details? The last I read it’s just used to merge multiple images - up to 8 or 9 images - to form the final
image. While I could see details getting lost or artifacts being added, I don’t think it can add actual “feature” details that don’t exist.
I wonder when the AI will hallucinate a gun into a black persons hand since the training black people often had guns? Hands moving fast are really blurry, so it has to hallucinate a lot, so it doesn't seem impossible. I could see that becoming a scandal of the century.
The novelty of things like instagram is wearing off. I see more people not bothering to pull out their phone. It's not just wanting to compete with the photos taken by narcissist on the internet, it's also just losing interest, and knowing things you share can and will be used against you.
This will be the next step in film “restorations” too.
A combination of ai models trained on high resolution textures and objects, models of the actors, and training from every frame of the movie that cal use the textures and geometry from multiple angles and cameras to “reconstruct” lost detail.
Oh man... I thought you were going to, "the stop signs and strip malls were how we discovered there were aliens on the Moon (and Mars) that look exactly like us!".
Of course they would have perfect skin and expertly applied eye-liner and lipstick as well.
Quite the equivalent (to me) to many kids preferring the taste of "strawberry yoghurt" compared to real strawberries, because it's sweeter and has enhanced taste. Except for photos.
I've seen this with CGI. CGI still looks awful somehow, but people about my age think it looks cinematic, and people a decade younger think it looks incredibly realistic.
Or, you know, Samsung sells ad placements in the enhanced images to do things like turn a can of Coke into a can of Pepsi, overwrite billboards in the background, etc.
Or better, the AI improves your shitty snapshots so they come out great. Every shot is beautifully framed, perfect composition, correct light balance, worthy of a master photographer. You can point your camera any old way at a thing and the resulting photo will be a masterpiece.
The details don't quite correspond to reality; to get the framing right the AI inserted a tree branch where there wasn't one, or moved that pillar to the left to get the composition lined up. But who care? Gorgeous photo, right?
And the thing is, I don't think anyone would care. You'd get the odd weird comparison where two people take a photo of the same place and it looks different for each of them. And you'd lose the ability to use the collected photos of humanity to map the world properly.
I think it's fascinating. Reality is what we remember it to be. We can have a better reality easily ;)
That could be fine as long as there is either a way to turn all that off (or better a way to selectively turn parts of it off) or a separate camera app available that lets you do that.
It's the future. Something hit your self-driving hover car and left a small dent. To get your insurance to pay for fixing the dent you have to send them a photo.
Your camera AI sees the dent as messing up the composition and removes it.
Your insurance company is Google Insurance (it's the future...Google ran out of social media and content delivery ideas to try for a while and suddenly abandon so they had to branch out to find new areas to try and then abandon). Google's insurance AI won't approve the claim because the photo shows no damage, and it is Google so you can't reach a human to help.
> Reality is what we remember it to be. We can have a better reality easily ;)
Cue Paris Syndrome, because expectations will also be of a better reality. Then you go somewhere, and eat something, and experience the mess that actually exists everywhere before some AI removed it from the record.
Something I learned long ago is that people typically don't want the truth, in general. They want fictional lies, they crave a false reality that makes them happy. Reality in and of itself, for most, is an utter drag if they're made constantly aware of it and dwell on it. When it comes to marketing, people eat up the propoganda techniques. They want to be fed this amazing thing even if it's not really all too amazing. They love that it tickles their reward center in the process.
This of course isn't always the case. When something is really important or significant people sometimes do want to know the truth as best they can. I want to know the car I'm purchasing isn't a lemon, I want to know the home I'm buying isn't a money pit, I want the doctor to to tell me if my health is good or bad (for some, under the condition the information is actionable), and so on.
When it comes to more frivolous things, for many, build the fantasy, sell them that farm to table meal you harvested from the dew drops this morning and hand cooked with the story of your suffering to Michelin star chef and how you're saving my local community by homing puppies from the local animal shelter with profits... even if you took something frozen, slapped it in the microwave and plated it and just donate $10 a month to your local animal shelter where you visited twice to create a pool of photos to market. For many, they want and crave the fantasy.
Progress made by science and tech has, for a brief fragment of history, established techniques and made practical, in some cases, to peel away all or at least some layers of fantasy away to reality. We started to pierce into cold hard reality and separate the signal of truth, as we can best understand it, from all the noise of ignorance and fantasy.
For many fantasy lovers, snakeoil salesmen, and con men, pulling away the veil of fantasy and noise has been a threat and there's been a consistent battle to undermine those efforts. The whole emergence and perpetuation of misinformation and recent "fake news" trends are just some of the latest popular approaches. We've been seeding our knowledge and information more recently with increasing degrees of falsehoods and pure fabrications.
Now, enter "AI," especially generative flavors. The same people who wanted to undermine truth are foaming at the mouth at the current ability to produce vast amounts of noise that in some cases are almost indistinguishable from reality from current techniques we have. Not only that, fantasy lovers en masse are excited at the new level of fantasy they can be sold. They really really don't care or want the truth. They really do just want "a good looking picture", "to make the summary interesting", or just see some neat picture. They don't care how accurate it is. Now people interested in the truth are facing a deluge of technologically enabled difficult to seperate noise production.
Is what I'm looking at close to reality? How many layers of noise are there I should consider when interpreting this piece of information? In the past, the layers used to be pretty managable, they were largely physical limitations or resource limitations to falsify the data to a point that couldn't be easily discerned. These days... it's becoming increasingly difficult to determine this and more and more information in various forms are leveraging more sophisticated and believable noise production. Technology has made this affordable to the masses and there are many parties with interest in setting the clock back to a world where the best story tellers are looked at as the oracles of modern time.
People often scoff at ChatGPT that it seeds or "hallucinates" to interpolate and extrapolate gaps of knowledge and make connections but it does so in a way that people like. It projects confidence, certainty, and in many cases it gives exactly what people want. To me, it's scary because it's providing a service the majority seem to want and creating an onslaught of noise that's more costly to debunk than it is to produce.
I think they're probably right about the AI-sharpening using specific knowledge about the moon... However, they are wrong about the detail being gone in the gaussian-blurred image.
If they applied a perfect digital gaussian-blur, then that is reversible (except at the edges of the image, which are black in this case anyway). You still lose some detail due to rounding errors, but not nearly as much as you might expect.
A gaussian blur (and several other kinds of blur) are a convolution of the image with a specific blur function. A convolution is equivalent to simply multiplying pointwise the two functions in frequency space. As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
Author tested for this by doing the experiment again with detail clipped into highlights, completely gone, model detail was added back.
> To further drive home my point, I blurred the moon even further and clipped the highlights, which means the area which is above 216 in brightness gets clipped to pure white - there's no detail there, just a white blob - https://imgur.com/9XMgt06
> I zoomed in on the monitor showing that image and, guess what, again you see slapped on detail, even in the parts I explicitly clipped (made completely 100% white): https://imgur.com/9kichAp
While I think this is a great test, I'm not really sure what that second picture is supposed to be showing. Kinda seems like they used the wrong picture entirely.
I watched the video and in this case the "recovered" detail is clearly natural to me. The original case does look like some kind of moon-specific processing, but this one with clipped highlights seems natural and can be achieved using classical CV.
> As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
Those rounding errors are very important though. The Gaussian function goes to zero very quickly and dividing by small numbers is not a good idea.
If your deconvolving a noise free version of the original that also doesn't have any saturated pixels (in the black or white direction) then you can get the pretty close to the original back. I don't think this applies here because the OP is taking a picture of a screen that shows the blurred version, so we've got all kind of error sources. I think the OP is right: the camera is subbing in a known picture of the moon.
It would be interesting to see what happens with anisotropic blur for example, or with a picture of the moon with added fake details (words maybe?) and then blurred.
> However, they are wrong about the detail being gone in the gaussian-blurred image.
Well yes, but he also downsampled the image to 170x170. As far as I know, downsampled information is strictly lost, and unrecoverable without an external information source (like an AI model trained with pictures of moon).
I'm too lazy to downscale it myself, so here's a 180x180 picture of the moon from WP [1]. This looks about the same as the Samsung result [2]. They are not getting the original detail, but they are getting the detail they should expect if Samsung simply deconvolved the blurred image.
>If they applied a perfect digital gaussian-blur, then that is reversible
Not true. Deconvolution is a statistical estimate. Think about it. When you blur, colors get combined with their neighbors. Statistically this moves toward a middle grey. You're compressing the gamut of colors towards the middle, and thus losing information. Look at an extreme case - 2 pixels of mid-grey. It can be deconvoluted to itself, to a light and dark grey, or to one black and one white. All those deconvolutions are equally valid. There's no 1-to-1 inverse to a convolution. If you do a gaussian blur on a real photo and then a deconvolution algorithm you'll get a different image, with an arbitrary tuning, but probably biased towards max contrast in details and light noise, since that what people expect from such tools and what most real photos have. But, just like A.I. enhanced images, it's using statistics when filling in the missing data.
But the AI should not have learned to apply a Gaussian deconvolution kernel. If anything it should be applying a lens-based bokeh kernel instead. A true lens blur does not behave like a Gaussian blur.
While the information might be recoverable, the information is not seen by the camera sensor. Hence I think the argument in the post stands. Some AI model/overlay magic is happening, pretending to display information the sensor simply did not receive.
That is mathematically true but not practically. Though indeed the Gaussian kernel has lots of zeros [1], in actuality, (a) the zeros themselves are at points, not regions, and therefore of little consequence, and (b) in practice the noise generated from reamplifying frequencies near these zeros can be minimized via techniques such as Wiener deconvolution [2].
A major problem with blur beyond rounding errors, say due to the optics being somewhat blurry due to manufacturing difficulties and tradeoffs for weakening assembly tolerance requirements (like wanting rotationally symmetrical optical surfaces, despite a rectangular shaped actively-used image focal plane (e.g. CMOS photodiode array), and potential for specializing the design to evenly light up _just_ that rectangle), is that the photon shot noise has a standard deviation equal to the square root of the photon count.
A smartphone sensor pixel has space for some low 4 digits number of electrons (created with some probability from photons, but that stochastic effect doesn't matter for anything a normal user would photograph) and typically should have a fixed 2~10 electron standard deviation from the analog-to-digital-converter (well, mostly the amplifiers involved in that process).
So if your pixel is fully exposed at a high 10000 electrons, and you √ that, you have 100 electrons stddev from shot noise plus worst case 10 electrons stddev from the readout amplifier/ADC.
If you have a dark pixel that only got 100x less light to only have accumulated 100 electrons, √ of that gives 10 electrons stddev of shot noise plus the same 10 electrons stddev readout amplifier/ADC.
The problem is that while you have an SNR of 5 with the dark pixel, when trying to deconvolve it out of a nearby bright pixel, even perfectly with no rounding errors (1 electron = 1 ulp/lsb in a linear raw format), you now have 100/110 = 10/11 ≈ 0.91.
That's far worse than the 5 from before.
This gets worse if your ADC has only the 2 electrons stddev instead of the 10 (about 2x worse here).
That's the reason why deconvolution after the photon detector is a band aid that you only begrudgingly tend to accept.
The trade-off just requires massively increased aperture/light gathering, likely negating your savings on optics.
> If they applied a perfect digital gaussian-blur, then that is reversible
Actually any noise distribution is frequently reversible if you know the parameters and number of steps. This is in fact how diffusion models work (there's even work of Normalizing Flows removing realistic camera noise). It is just almost impossible to figure this out since there are many equivalent looking ways. But we need to be clear that there is a difference between reversibility and invertibility. A invertible process is bijective, or perfectly recreates the original setting. A reversible process can just work in both directions and isn't guaranteed to be invertible. (Invertible means reversible but reversible doesn't mean invertible)[0]
I bring this up because even more complicated versions of bluring could be argued as not "faked" but rather "enhanced." A better way to test Samsung faking the data is to mask out regions. If the phone fills in the gaps then it is definitely generating new data. This can still be fine if the regions are small, unless we also want to call bilinear interpolation "faked" but I don't think most people would. This is why it gets quite difficult to actually prove Samsung is faking the image. I don't have a Samsung phone to test this though.
So basically I'm with you, and even a slightly stronger version of this
> It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
Edit: After reading other comments I wanted to bring some things up.
- The down scaling is reversible, but not invertible. We can upscale, reversing the process. But yes, there is information lost. But some data can still be approximated and/or inferred.
- The clipping experiment isn't that good. Honestly, looking at the two my brain fills in the pieces and they look reasonable to me too. Clipping the brightness isn't enough, especially since it is a small portion of the actual distribution. I did this on both the full image and small image and both are difficult to distinguish by eye from the non-clipped. Clipping below 200 seems to better wash out the bottom of the moon and remove that detail. 180 seems better though tbh.
The level of BS in this thread perfectly resembles the BS in religious-level audiophile discussions. A mixture of provably correct and provably incorrect statements all mixed together with common words used in uncommon ways.
> But yes, there is information lost. But some data can still be approximated and/or inferred.
Gaussian blur is essentially acting as a low pass filter. An IR filter does not strictly destroy information in the filtered spectrum components, but does attenuate their power.
Given a perfect blurred image, reconstruction is possible - however due to the attenuation, these high frequency components are ~sensitive~.
Apart from quantisation effects [you mentioned which limits perfect de-convolution], adding a little AW Gaussian noise(such as taking a photo of the image from across the room) after the kernel is applied obliterates high frequency features.
Recovery when noise is low (plus known glyphs) is why you should not use Gaussian blur followed by print screen to redact documents.
Inability to recover when there are artifacts and noise is [part of] why cameras cannot just set a fixed focus [at whatever distance] and deconvolve with the aperture [estimated width at each pixel] to deblur everything that was out of focus.
TLDR for readers, It is unlikely to recover sufficient detail via de-convolution here.
This is wrong. The blurred image contains only intensity information, but reversing the convolution in frequency space would require phase information as well. A simple Gaussian blur is not reversible, even in principle.
There is no "phase information" in the spatial domain. "Phase" is literally, where the pixels are on the screen.
Rather, reversing blur of any type is limited (a) by spatial decimation (a.k.a. down sampling, which is performed in the article), and (b) by noise/quantization floor, below which high frequency content has been pushed.
Hey all, it's the author of the reddit post here. First of all, let me say that I don't usually frequent HN, but the comments on here are of such high quality, that I might need to change that. I got semi-depressed on reddit, with people misattributing statements and, in general, not being overly, uh, skeptical :)
That being said, there were a few comments on here about gaussian blur and deconvolution, which I would like to tackle. First, I need to mention that I do not have an maths/engineering background. I am familiar with some concepts, as I've used deconvolution via FFT several years ago during my PhD, but while I am aware of the process, I don't know all the details. I certainly didn't know that the image that was gaussian blurred could be sharpened perfectly - I will have to look into that. In fact, I used gaussian blur to redact some private information (like in screenshots), and it's very helpful to know if I haven't redacted anything and the data is recoverable. Wow.
I would love to learn more about the types of blur that cannot be deconvoluted.
However, please have in mind that in my experiment:
1) I also downsampled the image to 170x170, which, as far as I know, is an information-destructive process
2) The camera doesn't have the access to my original gaussian blurred image, but that image + whatever blur and distortion was introduced when I was taking the photo from far away, (whatever algo they are using doesn't have access to the original blurred image to run a perfect deconvolution on)
3) Lastly, I also clipped the highlights in the last example, which is also destructive (non-reversible), and the AI hallucinated details there as well
So I am comfortable saying that it's not deconvolution which "unblurs" the image and sharpens the details, but what I said - an AI model trained on moon images that uses image matching and a neural network to fill in the data.
Thank you again for your engagement and your thoughtful comments, I really appreciate them, and have learned a lot just by reading them!
> In fact, I used gaussian blur to redact some private information
Absolutely never do that. I honestly don't understand why people still do, given that it's obvious that low levels of blur can be reversed why even risk guessing until what point someone might be able to recover anything? Just censor it, draw over it with an opaque tool, and save it in a format that won't store layers or undo history or something (the riskiest format being pdf).
If you don't like how that looks, the alternative is to replace the information and then blur it. They can unblur but will find an easter egg at best.
Personally, I censor instead of blurring a replacement, but I balance between low contrast and not hiding the fact that information was removed. A stark contrast distracts and looks ugly. E.g., for black text on a white background, I'd pick a light/medium gray (around the average black level of the original text, basically).
Should be rather easy to prove if Samsung is really able to „unblur“ an image in that way: use something else than an image of the moon as starting point and apply the same steps, i.e. down sizing and blur, then take a photo and see if it’s able to recover details.
I just wanted to say that the experiment at the end where you had half the moon and the whole moon was brilliant, and perfectly illustrated the problem in a single picture. If anyone hasn't seen that they should.
I took a max-(non-optical)-zoom photo of a rabbit in my yard a while back using an iphone, then further enlarged the result to see how it did - in the details it looked like an impressionistic painting of a rabbit, facing the camera and looking left. The actual rabbit was looking away from the camera and to the right. The eye and face were not visible.
Look at pictures on social media in recent months, pretty much every single image looks like this. If you zoom in, they're all inaccurate impressionist paintings. They only look good at original size.
It's absolutely painful and encourages me yet again to use my mirrorless camera even more.
At this point I wish we had legislation requiring a "turn off AI bullshit" option for any camera. Every time I've taken a picture of myself using the camera app built-in to WhatsApp, it blurs the shit out of my face in an attempt to hide blemishes or whatever, and (in a weird sort of reverse-vanity) it really annoys me that it gives the impression I care about looking un-blemished, which I do not. AFAICT, there is no way to turn this off.
While I understand what the author tries to say, I have to point out that ship has long sailed. Samsung just pushed it a bit too far and slapped a "scene optimizer" label on it.
AI has been used in "cell phone photography" for a few years, at lease since Pixel 2 where a mediocre sensor produced much better pictures than what people expect (maybe there are other players who did this even earlier). And every manufacturer started doing it, including Apple. Otherwise, do you think "night mode" is just pure magic? Of course not, algorithms are used everywhere.
How do you define "fake"? In podcasts, Verge editor Nilay Patel has asked various people "what is a photo", because the concept of a "photo" has become increasingly blurry. That is the question the author is asking, and people may have different answers from the author's.
> Otherwise, do you think "night mode" is just pure magic?
Night mode definitely uses some AI but most of the result is from stacking frames. Samsung here did not label it as a "scene optimizer". Their marketing just calls it Space Zoom. The only disclaimer they provide is "Space Zoom includes digital zoom, which may cause some image deterioration."
According to Samsung -- and I just confirmed it on my S22 under Camera -> Camera Setting -- it's called "Scene Optimizer"[1]:
[ Overview of moon photography]
Since the Galaxy S10, AI technology has been applied to the camera so that users can take the best photos regardless of time and place.
To this end, we have developed the Scene Optimizer function, which helps AI to recognize the subject to be photographed and derive the optimal result.
From Galaxy S21, even when you take a picture of the moon, ai recognizes the target as the moon through learned data, and multi-frame synthesis and deep learning-based ai technology when shooting. The detail improvement engine function that makes the picture clearer has been applied.
Users who want photos as they are without AI technology can disable the optimum shooting function for each scene.
Note that this behaviour is limited to scene mode, which has a moon shoot mode. You can always use the normal or pro mode where the pictures are not magically enhanced.
Is is ridiculous that OP consider this "cheating". Most people just want a nice picture and don't give a damn about AI.
Photo is an interesting word. It's meaning is clarified by other words, such as photorealism, photofinish. These words will (strictly) lose their meaning if photograph simply means image captured and processed by a device.
Curiously and revealingly, the political word photo-op stands alone in this photo- parade of words in the age of photo-imaginings. The universe does indeed have a sense of humor.
Using algorithms to take multiple pictures and stack them together is fine. The information is real, exists, and objective. People in the background won't (for example) suddenly be facing the other way because of the algorithm.
The problem is that AI isn't just interpolating data. It is wholesale adding extra data that simply doesn't exist. The person in the background is facing left, but the sensor couldn't possibly have captured that detail even after multiple images--it was a coin flip that the AI made.
The issue is that, like privacy, most people won't care ... until they do. By that time, it will be too late.
The software technology in the original pixel cameras were using multiple frames of varying exposure to allow for impressive dynamic range in images while still retaining colour and contrast. This is quite a difficult thing to do as requires precise understanding of what the 'edge' of an object is, and I think that is what AI was used for. This stacking technique is also used for night exposures.
I'm sure that they have started using AI to fill in details more recently, but this is just to point out clever use of multiple exposures and AI can help without faking detail.
This problem is not new. In 2019, Huawei introduced a special image processing feature in its smartphone camera app, the "Moon Mode" (opt-in). Missing details are added to the moon photos via machine learning inference from a pre-trained model. Huawei then started marketing these processed images as a showcase of its new smartphone's photography performance. In China, it was widely criticized by tech reviewers [1][2] as misleading, and "Moon Mode" became a running gag among tech enthusiasts for a while.
It seems that Samsung simply adopted the same tactic to compete...
On the Huawei "Moon Mode" controversy, one can even find a research paper [3] published in a peer-reviewed social studies (!) journal, Media, Culture & Society:
> This is where the controversy began: Chinese tech critic Wang’s (2019) posting on Weibo, the Chinese equivalent of Twitter, made quite a splash. In his post, Wang put forward a shocking argument: he said that Huawei’s Moon Mode actually photoshops moon images. He contended that, based on his self-conducted experiments, the system ‘paints in pre-existing imagery’ onto photographed takes, re-constructing details that are not captured in the original shots. Huawei immediately refuted these claims, stressing that the Moon Mode system ‘operates on the same principle as other Master AI modes that recognize and optimize details within an image to help individuals take better photo'
This couldn't be the same Huawei who has been caught repeatedly using DSLR stock photography in their marketing materials, while claiming the images were taken by their smartphones.
It would be so fun to hook a phone up to a telescope and take a picture of, say, Jupiter, and see if it overlays the ringed planet with the moon's characteristics.
My hypothesis is that the neural network was trained on a lot of labeled photos, so somewhere inside the network, when you see the moon, it has some moon=0.95 confidence number, and whatever label has the highest confidence, it tries to bring it up to 1.0 akin to how deepdream makes images of spaghetti have more dog faces. Samsungs marketing department interprets that as technically enhancement of images and not faking the moon specifically. So perhaps if it sees Jupiter, it will try to make it more jupitery.
This is just make hipsters get into old point and shoot digital cameras... good thing I kept my Canon A540.
All the subtle trickery manipulation that the smart phone's doing to reality is concerning. Smoothing people's faces, making their eyes pop, enhancing the shit out of the colours, and now plopping fake objects overtop of the real ones.
Future concerns of this technology should range from a low-key disconnect from reality, to the complete inability to photograph certain objects or locations.
Imagine dusting off a 30 year old digital camera, finding some AA batteries to put in it, snap a selfie and then realizing just how ugly we all are and how washed out the polluted world actually looks without a bunch of narcissism-pandering enhancements.
I'm 40 and did the same thing, bought a cheap analog camera with black and white film to shoot fun photos on my birthday party :) The film is still at the lab, but I guess it's worth the wait.
I love my Canon AE-1 but dang it's expensive to actually use.
The G9x Mark II, Sony RX100, Panasonic LUMIX and similar 1" sensor cameras are awesome though and I don't think they've gotten too crazy with computational photography. I imagine some color processing modes might be doing a bit of work though.
I regularly take photos of text etc because I am not going to remember it. If a photo of a config password is AI fucked into showing the wrong digits, there’s a real problem.
> I downsized it to 170x170 pixels and applied a gaussian blur, so that all the detail is GONE. This means it's not recoverable, the information is just not there, it's digitally blurred
Strictly speaking, applying a Gaussian blur does not destroy the information. You can undo a Gaussian blur with a simple deconvolution, which is something I would expect even a non-AI image enhancement algorithm to do (given that, you know, lenses are involved here).
I'd like to see what detail can be "recovered" with just the downsizing, which DOES destroy information.
Well the op did downsize so details had to be reconstructed. Also the noise from having the image being projected through a screen and then a retaken through the camera sensor means that it isn't just your standard perfect convolution.
if you downsize an image to 170x170px and then blow it up so it's visible to a camera from across the room without any sort of blurring, it's not going to look like anything and the camera's object detection won't recognize it as the moon - it's just going to look like a huge pixel grid.
I am not quite so confident. I would like to see an experiment to test how badly you can distort an image of the moon before the AI stops recognising it.
I photoshopped one moon next to another (to see if one moon would get the AI treatment, while another would not), and managed to coax the AI to do exactly that.
I picture someone 20 years from now trying to find out what their parent really looked like when they were young. The obviously smoothed-out face filters are already giving way to AI-powered homogenization. And the filtering is moving deeper down the stack from the app to the camera itself. There will be no "original".
Note that the author decides that Samsung's photos are "not fake", in the sense that they were not doctored after being taken with the phone. However, the article decisively proves that they're being heavily doctored in-camera.
Another test would be to shoot RAW + JPEG if the camera supports it. A true RAW image would reveal what the sensor is actually capturing.
> For example Apple can save ProRaw format which DOES include the postprocess magickery, but uncompressed and having more detail for post. BUT, it also provides apps with true RAW, which is true (or very close to true) raw.
Thank you for that detail. IMO, that makes "ProRAW" a really unfortunate naming decision on Apple's part. And I think you're saying that if you shoot in RAW with, say, Halide, the result will be an actual RAW (pre-demosaic) file in DNG format.
I have a similar photo done on S22 ultra on March 6 this year, and neither look like your (position of lower right mega crater but also the rest). So its not simple 'photoshop-into-predefined-nice-image'.
I can clearly see that most folks here don't actually own discussed devices (which is fine, its US-based HN, a bastion of iphone and many Apple employees dwell here and uncritical appreciation of Apple is very evident in every single related thread). I've used its 10x zoom extensively over more than a year, it simply blows all other phones away easily for that kind of situation (more than those rather weak 3x zooms available everywhere). Family photos, wild animals, nature, anything you want to come closer, otherwise the scene is tiny dots in the center like on other phones. It works really well for what it is, with obvious unavoidable physical limits.
Overall this phone made me put my fullframe Nikon D750 away on a day I bought it. I took it 'just to be sure' on vacation to Egypt last year, didn't touch it a single time. Most often it doesn't produce strictly as good images but a) they are good enough to be viewed on phones side by side easily, basically as good as fullframe there and sometimes even much better, ie handheld photos in the night of dark scenes, fullframe is utterly lost without tripod, and b) it weights 0 and takes 0 extra space (and cost 0 instead of many thousands for modern camera with big sensor), since I have phone with me always anyway.
Tried exactly the steps as author of article, couldn't reproduce it a bit, tried various mega zooms, his various original photos, dark room etc. Blur remained blur, nothing added. I mean at this point everybody acknowledges any decent phone is painting quite a bit (ie iphone taking other side of bunny than reality, thats a fine example) and I am sure Samsung is doing their part as they have the literal android flagships.
Orientation is because photos were taken months apart.
This is to get an idea of quality, with the APS-C camera having a much larger (26x area) sensor and better optics to work with.
My impression is... S21 Ultra "space zoom" is, at best, a good party trick. But if you zoom in, the quality is still nearly garbage. Not an objectively "great photo of the moon."
I'm not very convinced. It looks like they are doing some processing, maybe special to the moon but it looks more like some form of sharpening or contrast boosting than adding detail. In all of the examples it seems that there is information in the original (dark spots) that are getting boosted.
It would be interesting to see this tried on a source image that isn't the moon. Just white with a few dark spots. Does it actually add in completely new craters, or just where there are existing smudges? Or do something like half of a moon photo and have white, does it add craters to the white side?
The OP tried to do this by changing the contrast but I failed to see any craters appearing where there wasn't already dark spots in the source photo.
It does seem strange that the OP is using an image of the moon to start and that they don't provide a still shot of the one where they modified the brightness levels to cause clipping. It doesn't really "drive the point home" as claimed.
Of course the answer to these may be that you need something moon-like enough to trigger the moon optimizations. But if that is the answer it would be interesting to see something that comes right up to the threshold where it either snaps in and out of these optimizations or two very similar images produce widely different results.
> In all of the examples it seems that there is information in the original (dark spots) that are getting boosted.
That was the point of the Gaussian blur. By blurring the source image of the moon before taking its picture, there was no information. The Gaussian blur destroys fine detail, by design.
The image enhancement applied by the Samsung phone is adding detail where there was none originally – not just detail that might be buried in the optics somewhere, but not there at all. It involves a computational model of what the target (here the moon) "should" look like and guesses at what it thinks are the blurry bits.
No, Gaussian blur does not "destroy" fine detail. It is still there it is just that the "volume" of it is turned down. You can put all the fine detail back again by applying the inverse filter.
Gaussian blur is reversible with deconvolution - however that is almost certainly not what is happening here, as it’s fairly computationally expensive.
But I don't see it adding any detail over the blur. It really just looks like it is boosting contrast a bit on top of the blur. I don't see anything like ridges or defined features appearing.
Does it enhance the moon only, or the Taj Mahal, Grand Canyon and Montblanc, too?
How long until your phone detects the subject based on geo localization and replaces your shot with a stock image of the subject from an immense database of selected, professional-looking pictures?
DeepL.com translation of relevant bit (there's some irony in using ML to be able to read text about ML):
> However, the lunar photography environment has physical limitations due to the long distance to the moon and lack of light, so the actual image output from the sensor at high magnification is noisy and not enough to provide the best image quality experience even after compositing multiple images.
> To overcome this problem, Galaxy Camera applies an AI detail enhancement engine (Detail Enhancement technology) based on deep learning at the final stage to effectively remove noise and maximize the details of the moon, resulting in bright and clear moon photos.
It's not just this one thing, though. It also describes moon detection resulting in: setting brightness, using optical stabilization, "motion sensor data and video analysis"¹ for stabilization (VDIS), and fixing the focus at infinity. The "AI" magic is then described as being a detail enhancer, which could mean anything from {generically decreasing blur and improving contrast} to {applying a model that was purposefully overfit on the moon} (most likely it's the latter). Either way it's fake, obviously, but why do all the effort and make compromises to get a good moon shot if you're going to replace it anyway?
> If you shoot the moon in the early evening, the sky around the moon will not be the color of the sky you see, but rather a black sky, which is caused by forcing the image brightness down to capture the moon clearly. [I think this translation should have read "picture brightness"; elsewhere it also says "screen brightness" so I suspect the Korean word for "picture" is ambiguous]
So you can't shoot anything near the moon, like if someone is holding the moon up with a hand or something, presumably that would be all black. It's apparently still relying on the sensor to get most of the way there and using ML for the last leg.
Imo the feature should spawn a warning on screen "Details of moon filled in by computer and may differ from reality" with buttons for [ok] to quickly dismiss as well as [don't show again]. Then you can't not know that your images are being faked and it's not disingenuous, while most people would still appreciate the better quality because it'll be/work fine in 99% of cases.
I guess there is some AI algorithm that does zoom as postprocessing. That AI knows the moon so it can fill in the blanks and compensate for a (relatively) crappy sensor.
So in the future, there are either cameras that can see what others have seen before, and those that can truly capture new, true, detail (true as in, without filling it with estimations)
Idea for further testing to prove the hypothesis beyond reasonable doubt:
What happens if you manually add a few additional artificial details to your copy of the image -- like a trap street in maps. Does the camera slip up and show flawless moon instead?
I have a few questions: Is this specific for the moon or is the AI generally sharpening around dark areas? If it is specific to the moon then how many common objects does the camera recognize? If you did the same test with a picture of a stop sign or a popular car would the results be similar? Also is this processing done on the camera or does this require an internet connection and work on Samsungs end? I have known about AI image enhancement for years but the idea of recognizing then re-texturing common objects is something I had never considered.
The Gaussian blur they applied is theoretically reversible in a continuous function and infinite precision. In a discrete function (like a image in the computer) and only a few dozens of bits it's not 100% reversible, but it can be partially undone and get a sharper image (that is not as sharp as the initial image).
But it isn't seeing, right? I mean, seeing a digital image of a thing is totally different from seeing the actual thing.
A digital image is a representation/abstraction/map kind of thing. And then a little bit of mental shorthand inside your head makes a connection and declares the thing and the representation-of-the-thing to be one and the same.
But it isn't the same thing. Not by a million miles.
It's creepy how deceptive phone cameras are. I've noticed in pictures that I've taken that certain elements have been postprocessed to look better than they actually are.
This is a cool and useful tech, but it obviously needs a good marketing story to avoid looking creepy. To think that no one will notice just makes it worse.
A gaussian blur is not a good test. It does not technically remove all the information from the image. As such this test can't distinguish between unblur or actual moon-pasting.
Binning the image, or cropping it in Fourier space, would be a better test.
I think it’s even more damning seeing this happen on the phone near the bottom of the article. There’s no way a phone camera sensor can capture all those raw values of the moon with 45X digital zoom with a phone telephoto lens (you can only see so far with 5 mm of packaging). From a digital sensor point of view, it probably sees a white blob and fires up the AI when it has a contrast and lighting similar to the moon
What would be even funnier (and more obvious, once people caught on) would be if the phone used its orientation sensors to only activate "Moon Mode" when pointed at the approximate current location of the moon. :)
Wait ... a Gaussian blur? That doesn't remove ANY information from the image. It looks blurred to our eyes but the same information is all there.
The information is absolutely not gone.
Does it state that it hallucinates craters that were not there in the original, or is it possible the filters simply did an FFT, adjusted the power spectrum to what we expect of a non-blurry picture, hence inverting the Gaussian blur?
EDIT: Note that a deblur of a smooth but "noisy" image can cause "simulation" or "hallucination" entirely without AI. Could be any number of things causing an output image like that (wavelet sharpening, power spectrum calibration, ...). Even if the information isn't recoverable as such, a photo of a Gaussian blur has an unnatural power spectrum that could easily "trick" conventional non-AI algorithms into doing such things.
Especially since the only thing I see in the output is "more detail" (i.e. simply a different power spectrum than the author expected..)
Unless you’re using floating point numbers and have padded the image to infinity, it’s literally untrue that applying a Gaussian kernel doesn’t throw away at least some (if not a lot of) information. I think what you intended to say is the blur doesn’t remove “all” the information. Not to mention he downsized the image (likely using bilinear not even bicubic ) further throwing out a ton of information. Not to mention he then showed the image on a display and took a photo from across the room. For all practical purposes the technically incorrect pedantic statement does not apply even if you correct it.
Looking at the images, it will be fantastical if not a violation of information theory if the phone didn’t use prior information about how the moon looks to create the photo.
Source: spent way more months than I would have preferred calculating CRLB values on Gaussian blurred point source images.
It also happened after downsampling (aka removing information) of the image. And when parts of the image was clipped to white (aka no information at all). So your point doesn't absolve Samsung of guilt as you believe.
Yeah the author glossed over this a bit in my opinion. In infinite precision math you're correct, but at some point the signal in those higher frequencies is going to be reduced below the precision of the storage data type, never mind the dynamic range of the monitor and camera he's using.
It seems clear the author does not have knowledge on the subject, more than glossing over -- the article even emphasises that information is removed by the blur (100% wrong). I agree it may not have destroyed the experiment entirely, but it does mean the experiment was conducted without knowledge of basic signal processing and I would prefer a more through study or analysis before drawing conclusions...
Well it is a conclusion written without knowledge of signal analysis.
The input to the algorithm will be an image with a power spectrum that isn't natural, . It would be very natural even without "AI" to attempt to "deblur" when faced with such a power spectrum.
A deblur of noise can cause "hallucinations" with much simpler reasons than AI beong involved.
Could it not e.g. be doing some wavelet transforms or FFTs and automatic power spectrum calibration?
How it is possible that people belived that these images are not artificially enchanced? Like it is obvious, there is no way a phone sensor can take such detailed photos of the Moon. I am surprised this was even a discussion.
The most impressive thing is how dumb the Reddit comments are. Incorrect factual claim after incorrect factual claim. Confidently stated and widely upvoted.
Was it always like this? Were we, people always like this?
Have you read comments here? Confidently exclaiming that downscaling, gaussian blur, and taking a picture of screen showing that from far away can easily be restored by deconvolution...
> Confidently exclaiming that downscaling, gaussian blur, and taking a picture of screen showing that from far away can easily be restored by deconvolution...
I don't think HN (or Reddit) posters are arguing that deconvolution was actually being used, or that it would be practical to do so. Instead, they're just nitpicking about the technical correctness of this particular statement - "the (ir)reversibility of a perfect digital gaussian blur" - even if it doesn't affect the conclusion of the experiment.
It's just the standard nerd-sniping as seen in all tech communities... If the debate on whether you can "reverse a photo of a gaussian blur taken by a camera from the monitor" became hot enough, eventually someone may even spend a weekend to code a prototype to show how it actually works better than most people's imagination, with the solely purpose to win an argument on the Internet.
Nearly every forum is like that. It may even be related to Dunning Kruger, in that people who overestimate their knowledge are obviously going to feel knowledgeable on more things.
That said, Reddit is probably the worst place for it. I rarely read comments anymore there because of how 'dumb' they are. Worse, I feel like Reddit is like some self trained botnet at times. Go on any post, say 'blood is thicker than water', and without fail someone is going to tell you the original is 'blood of covenant is thicker than water of the womb.' This is of course incorrect, and the only place I've ever seen it is...Reddit. So people posting ridiculous things learning from people posting ridiculous things.
It happens here though, too, with the main difference being someone will usually correct you, and not be downvoted for doing so...
First - I see that you don't have any audiophile friends! Many sound geeks dislike the sound of compressed audio.
Second - there's nothing wrong with an AI 'enhancement' option. It seems useful! But you should admit what you are doing and have an off switch. Imagine if all of the studio masters for the past 20 years were actually in lossy mp3 because the hardware switched silently?
The changes the AI makes are perceptually noticeable though when compared to source.
No one complains about image or video compression either if the quality is good enough - because it’s not perceptually noticeable vs uncompressed; people can do A/B testing to be sure.
I tinkered with the idea to let a app use your personal gallery as training data and create a "barney camera"(himym) where 100% of photos of you and your friends would always be perfect, it would be impossible to capture a bad photo.. same here, its the moon just perfect .. who cares its not like it changes ..
This question is less hypothetical than it seems. In recent years, there has been amazing breakthroughs in low-light photography with machine learning, such as [1]. It's not hard to imagine that it would be applied to future surveillance cameras.
I imagine that in most cases, the "AI" would work just fine at guessing things. But important edge cases exist, including the possibility of adversarial machine learning spoofing attacks to create false images. Imagine faking a crime scene this way, it would something straight from cyberpunk fiction.
Every few months there's a new clown on reddit that doesn't know anything about photography or gaussian blur and tries to expose smartphones for using AI based sharpening and interpolation.
Oh nooooo this phones photos come out much sharper than my peanut brain expected!!! Fraud!!!!!!!!!!
Has anyone found a decent use of the 100x zoom on these phones? Could it be that the sheer hype it causes compensates for the extra cost of putting it on the phone? Like, even if nobody uses it it stills turn an overall profit because of the marketing
Not sure about the current situation, but a couple years ago could receive tons of updoots on Reddit for posting a video how you zoom eg from one side of the harbour to see something on the other side.
In this regard, the AI is acting just like the human brain - adding details that may or not be present in what the viewer is seeing. Details that the viewer expects to see, so the brain delivers against this expectation.
Am I supposed to be convinced by this? Cause I'm not.
It looks more like moon mode is assembling fake detail out of shot noise from a photo being taken in the dark indoors. That's enough to cancel out his blurring and all that.
Even if it is guided upsampling with a "this is a moon prior", so what? That's not "fake", it's constrained by the picture you took.
I think you misread the article. What they did was:
1. Use Photoshop to blur a photo of the moon, destroying detail
2. Use a Samsung camera to take a photo of the blurred photo of the moon
3. The camera somehow emits a crisp photo of the moon, including the detail that was destroyed in step 1
It seems like the camera AI detected 'this is a photo of the moon', and used its knowledge of what the moon looks like to add the detail back in. Where else could the camera have gotten the detail from?
Imagine this future:
Sensor quality in phones goes down, AI makes up for it because good sensors are expensive, but compute time in the cloud on Samsung owned servers is cheap. You take a picture on a crappy camera, and Samsung uses AI to "fix" everything. It knows what stop signs, roadways, busses, cars, stop lights, and more should look like, and so it just uses AI to replace all the textures.
Samsung sells what's on the image to advertisers and more with the hallucinated data. People can't tell the difference and don't know. They "just want a good looking picture". People further use AI to alter images for virtual likes on Tiktok and Insta.
This faked data, submitted by users as "real pics in real places" is further used to train AI models that all seem to think objects further away have greater detail, clarity, and cleanliness than they should.
You look at a picture of a park you took, years before, and could have sworn the flowers were more pink, and not as red. You are assured, by your friend who knows it all, that people's memories are fallible; hallucinating details, colors, objects, sizes, and more. The image, your friend assures you further? "Advanced tech captured its pure form perfectly".
And thus, everyone will demand more clarity, precision, details, and color where their eyes don't remember seeing.
Now, imagine this future:
You got a friend, spouse or someone close that has hundreds of pictures of you on their phone. Their phone has a "AI chip" that is used to finetune the recognition models and photo models with your AI library. Like Google Photos tags images of people you know, so does the model. It also helps sharpen images - you moved your head in an image and it was a bit blurry, but the model just fixed it, because like the original model had for the moon, it has hundreds of pictures of you to compensate.
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Good point.
For the long time digital cameras embedded in EXIF metadata about conditions on which the photo was made. Like camera model, focal length, exposure time etc
Nowadays this metadata should be extended with description of AI postprocessing operations.
5 replies →
The far greater concern is far more mundane.
Photos taken by cell phone cameras increasingly can't be trusted as evidence of the state of something. Let's say you take a picture of a car that just hit a pedestrian and is driving away.
Pre-AI, your picture might be a bit blurry, but say, it's discernible that one of the headlights had a chunk taken out of it; it's only a few pixels, but there's obviously some damage, like a hole from a rock or a pellet gun. Police find a suspect, see the car, note damage to the headlight that looks very close, get a warrant for records from the suspect, find incriminating texts or whatnot, and boom, person goes to jail for killing someone (assuming this isn't the US, where people almost never go to jail for assault, manslaughter, or homicide with a car) because the judge or jury are shown photos from the scene, taken by detectives in the street of the person's driveway, and then from evidence techs nice and close-up.
Post-"AI" bullshit, the AI sees what looks like a car headlight, assumes the few-pixels damage is dust on the sensor/lens or noise, and "fixes" the image, removing it and turning it into a perfect-looking headlight.
Or, how about the inverse? A defense attorney can now argue that a cell phone camera photo can't be relied upon as evidence because of all the manipulation that goes on. That backpack in a photo someone takes as a mugger runs away? Maybe the phone's algorithm thought a glint of light was a logo and extrapolated it into the shape of a popular athletic brand's logo.
1 reply →
The recent kyle rittenhouse trial had an element that hinged on whether apple's current image upscaling algorithm uses AI, and hence whether what you could see in the picture was at all reliable. The court system is already aware of and capable of dealing with these eventualities.
5 replies →
I thought it was really funny in the 1980s that people in medical imaging were really afraid to introduce image compression like JPEG because the artifacts might affect the interpretation of images but today I see article after article about neural image enhancement and it seems almost no concern that a system like that would be great at hallucinating both normal tissue and tumors.
So far as law and justice goes it is the other way around too. If it is known to be possible that cameras can hallucinate your identity, it won't be possible to use photographic proof to hold people to account.
It seems fairly easy to bake a chain of custody into your images. Sensor outputs a signed raw image, AI outputs a different signed “touched up” image. We can afford to keep both in this hypothetical future; use whichever one you want.
Once generative AI really takes off we will need some system for unambiguously proving where an image/video came from; the solution is quite obvious in this case and many have sketched it already.
13 replies →
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." [1]
If any of you young folks haven't watched Ghost in the Shell, just close this tab and do that.
[1] https://ghostintheshell.fandom.com/wiki/Laughing_Man
8 replies →
Obviously someone who has good enough position to take semi-clear photo and who knows you so well, that has phone full of your face, will not recognize you directly, but will be convinced that you are robber after looking at photo. At this point we can go full HN and assume that you will be convinced anyway, because judge is GPT-based bot.
This "future" is present in current Pixel lineup btw. Photos are tagged as unblured, so for now you can still safely take a selfie with your friends.
They should call the chip XeroxAI.
https://news.ycombinator.com/item?id=29223815
imagine you want to "scan" a document using camera app like many people do, and ai sees blurry numbers and fixes then for you. when will you notice that some numbers even that look clear are different than on original document?
3 replies →
AI-based image generation is surely already good enough that a single digital photo can't count as evidence alone. But your scenario doesn't make much sense to me - are you suggesting AI will have reached a point it's stored and trained on images of almost everyone's faces, to the point it could accurately/undetectably substitute a blurry face with the detailed version of an actual individual's face it happens to think is similar? I'd be far more worried about deliberate attempts to construct fake evidence - it seems inevitable that eventually we'll have technology to cheaply construct high-quality video and audio media that by current standards of evidence could incriminate almost anyone the framer wanted to.
1 reply →
> Congratulations, you are now a robber.
Yeah, but in the future the government will know your precise location, all day, every day, so at least you'll have an alibi.
1 reply →
Whether zooming in on an image on iPad adds "extra" details was already a contentious discussion during Kyle Rittenhouse trial. The judge ultimately threw that particular piece of evidence out, as the prosecution could not prove that zooming in does not alter the image.
Now imagine that we can use a future full homomorphic encryption and train models without revealing our private data.
Obligatory throwback to the Xerox incident: https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
One day, that person witnesses a robbery. They try and take a photo of the robber, but the algorithm determines it was you on the photo and fixes it up to apply your face. Congratulations, you are now a robber.
Sounds like pretty standard forensic science, like bite marks and fingerprints.
Basically this.. As "neat" as AI "improvement" is, I don't think it has any actual value, I can't come up with any use-case where I can accept it. "Make pictures look good by just hallucinating stuff" is one of the harder ones to explain, but you did it well..
Another thing, pictures for proof and documentation, maybe not when they're taken but after the fact, for historical reasons, or forensics.. We can't have every picture automatically compromised as soon as it's taken. (Yes, I know that photoshop is a thing, but that's a very deliberate action, which I believe it should be)
I think the main use case is "I'm a crummy photographer and all I want is something to remind me that I was there" and "Look at my cat. Look! Look at her!"
That's me. I'm a lousy photographer, as evidenced by all of the photos I shot back when film actually recorded what you pointed it at. My photography has been vastly improved by AI. It hasn't yet reached the point of "No, you idiot, don't take a picture of that. Go left. Left! Ya know what, I'm just gonna make something up," but it should.
I imagine there will remain a use case for people who can actually compose good shots. For the remaining 99% of us, we'll use "Send the camera on vacation and stay home; it's cheaper and produces better pictures" mode.
8 replies →
Interestingly enough one of the reason Sonys flagships perform really badly in comparisons is because they are weak at computational photography. So even when the sensor is great it looks too real, which people don't like.
1 reply →
How about using AI for sensor fusion when you have images from multiple different kinds of lenses (like most smartphones today)? I was under the impression this was the main reason why AI techniques became popular in smartphone cameras to begin with
1 reply →
Good for situations where you aren’t expecting or care about realism in this detail. AI hallucinations will be amazing for entertainment, especially games.
1 reply →
Removing noise from low lights pictures, or removing motion blur from shaky hands. Lots use cases for “ai” or computational photography.
> We can't have every picture automatically compromised as soon as it's taken.
Isn't it a good thing for privacy?
1 reply →
I genuinely can’t recall people saying “hallucinate” with any regularity - in the context of “AI” - until people started talking about ChatGPT.
So, we’ll see what people say in a year.
4 replies →
I don't already fully trust the images, audio and videos I take with the phone.
I'm working close to HW and I actively use the camera/picture and videos for future reference and debugging. It's small, fits in your pocket, and the bloody thing can record at 240fps to booth!
Until you realize there's so much post-processing done on the images, video and audio you can't really trust and can't really know if you can turn it all off. The reality is that if you could, you'd realize there's no free lunch. It's a small sensor, and while we had huge improvements in sensor and small lenses, it's still a small sensor.
Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
In the end, they're consumer devices. "Does it look good -> yes" is what thrums everything in this market. Expect the worst.
> Did the smoothing/compression remove details? Did the multi-shot remove or add motion artifacts you wanted to see? Has noise-cancelling removed or altered frequencies? Is the high-frame rate real, interpolated, or anything inbetween depending on light just to make it look nice?
This has been true of consumer digital cameras for 25 years. It's not new to or exclusive to smartphone cameras. It's not even exclusive to consumer cameras as professional ones costing many times more also do a bunch of image processing before anything is committed to disk.
4 replies →
I don’t know about android, but at least with my iPhone I’m pretty sure there are apps that can capture raw sensor data. Additionally I do have the ability capture Apple ProRAW format at of the photos. I don’t actually know if these images are still processed though.
1 reply →
I can imagine there will be a lot more “ghost” pictures like this one: https://www.reddit.com/r/Ghosts/comments/11jvwy4/after_years...
as AI tries to infer images of people where they aren’t really present.
I don't know if you even need AI for this.
"You just took a picture of the Eiffel Tower. We searched our database and found 2.4 million public pictures taken from the same location and time of day. Here are 30,000 photos that are identical to yours, except better. Would you like to delete yours and use one of them instead?"
royalty free I'd assume you meant as well, or are you pitching a new SaaS model for stock photos?
13 replies →
There was a pretty neat Google project a few years back that showed time-lapse videos of buildings under construction created entirely through publicly posted images that people had happened to take at the same spot over time.
I wonder if that'll ever cause legal problems in the future. Sorry, that photo someone took where the accused was in background at a party some years ago? He was kinda blurry and those facial features have been enhanced with AI, that evidence will have to be thrown out. Or maybe the photo is of you, and you need it as an alibi..
This is actually exactly what happened during the Kyle Rittenhouse case. A lawyer for the defense tried to question video evidence because of AI being used to enhance zoomed shots.
8 replies →
"Good sensors are expensive"-fun-fact: Mid-range CCTV cameras often have bigger sensors (1/1.8" or 1/1.2") and much faster lenses than an iPhone 13 Pro Max (1/1.9" for the main camera). The CCTV camera package is of course far bigger though. But still kinda funny in a way.
Edit: And the lenses on these are not your granddads computar 3-8/1.0, either. Most of the CCTV footage we see just comes from old, sometimes even analog, and lowest-bidder installations.
Bruce Sterling I think had a story in that direction. A polaroid camera producer would develop photos which would've been algorithmically enhanced so that their clients consider themselves better photographers and their cameras superior. I'm regularly updated for it for the last few years when cameras are more and more their software.
Edit: fixed the author's name. Cannot find the exact story though.
This has in some ways been happening for decades. There are a few countries where the way to take a good portrait of a person is to over expose the photo, so skin tones are lighter. People bought the cameras and phones that did this by default (by accident or design in the 'portrait mode' settings). They didn't want realism.
This is just a progression of the nature of our human world - we have been replacing reality with the hyperreal for millennia, and the pace only accelerates. The map is the territory. Korzybski was right, but Baudrillard even more so.
But it won’t end there.
Eventually people won’t care much for clarity and precision, that’s boring. The real problem is that everything that can be photographed will eventually have been photographed in all kinds of ways. What people really want is just pictures that look more awesome, in ways other people haven’t seen before.
So instead, raw photos will be little more than prompts that get fed to an AI that “reimagines” the image to be wildly different, using impossible or impractical perspectives, lighting, deleted crowds, anything you can imagine, even fantasy elements like massive planets in the sky or strange critters scurrying about.
And thus, cameras will be more like having your own personal painter in your pocket, painting impressions of places you visit, making them far more interesting than you remember and delighting your followers with unique content. Worlds of pure imagination.
You can already do that with AI art generators right now. Can either generate images from scratch using prompts or enhancre existing images.
3 replies →
I like the story, but I think people will notice pretty quickly as almost everyone reviews their photos right after taking them (so they can compare them with what they see in reality)
True, its just a fun story. This reddit post makes it clear, though, that while people will review the images carefully, they may still not be able to accurately determine differences.
Just take the story above with one more minor step: You snap a pic of the park, briefly glanced at it to make sure it wasn't blurry (which the AI would have fixed anyway) or had an ugly glare (it did, the AI fixed it) or worse a finger (the AI also fixed that).
You're satisfied the image was captured faithfully and you did a good job holding your plastic rectangle to capture unseen sights. You didn't look closely enough to notice all the faked details, because they were so good.
This fake moon super enhance? It already proves people will fall for it. I could easily see people not realizing AI turned the flowers in the picture more red, or the grass just a little too green, etc.
18 replies →
Doubtful - they’ll just think they’ve taken a great photo because they’re a skilled photographer, and they won’t be shy about telling you so.
I guess I havent noticed that people do that for things other than selfies.
I generally just burst-mode-scan an area or scenery location and later that night, or when I add to Strava or wherever, I have an old school contact sheet (but with 60-80 images per thing) to look though. Then narrow it down to 5-10, pick the one or two I like best and discard the rest.
It's sorta like this already, in the _present_ - people post photos with filters all the time, smart phone cameras color-correct and sharpen everything with AI (not just Samsung's). It'll just become more and more commonplace
The problem is that this particular AI enhancement was not advertised as such. Also, in the linked article it was putting moon texture on ping pong balls, which seemed like overzealous application of AI. Samsung could have marketed it as "moon enhancement AI" or something like that, which would be more honest.
My worry about these features becoming commonplace is that if everyone just leave those features enabled, we would end up with many boring photos because they all look similar to each other. The current set of photo filters, even though they seem to be converging on particular looks, at least don't seem to invent as much detail as pasting a moon that's not there.
1 reply →
I still argue that my Galaxy Note 8 took cleaner pictures in general than my Galaxy Note 20. Everything feels overly processed, even in "pro" mode with all processing settings turned off.
"Do you believe for your own eyes, or what I say!?"
The future is BigCorp's AI will censure your photo when you try to take picture like on tiananmen square massacre.
I want a photo, not even sharpend.
I’ve always thought this was the final outcome of all AI; a feedback loop. Same when ChatGPT starts using things ChatGPT wrote itself as references to train itself.
We already have people demanding higher definition televisions to watch AI-sharpened 4K restorations of old films whose grain and focus would annihilate any details that small worth seeing.
There's an arms race between people adding nonexistent details to old films and people manufacturing televisions with dense enough pixels to render those microscopic fictions. Then they lay a filter over it all and everything becomes smooth gradients with perfectly sharp edges.
People want this. It's already happening. There was a post on the Stable Diffusion reddit where someone ran a picture of their grandparents through it to colorize and make it "look nicer". But it made significant changes to their clothes and hallucinated some jewelry they weren't wearing, along with some subtle changes in the faces. It's not real anymore, but hey it looks nicer right?
ControlNet already fixed that, you can constrain it to change colors and nothing else.
What you're imagining is Hyperreality from Simulacra and Simulation and has been happening since the invention of the television, and later the internet.
AI will accelerate this process exponentially and just like in The Matrix, most people will eventually prefer the simulation to reality itself.
This is my exact worry with things like chat gpt polluting the scrapable internet. The feedback loop might eventually ruin whatever value the models currently have by filling them with incorrect but plentiful generated nonsense.
I was thinking of a scenario. My children are adults and browsing photos of themselves as children. They come across a picture of the family on a vacation to the beach. They dimly remember it, but the memories are fond. They notice they are holding crisp ice cold cans of Coca Cola Classic (tm). They don’t remember that part very well. Mom and dad rarely let them drink Coke. Maybe it was a special occasion. You know what, maybe it would be fun to pick up some Coke to share with their kids!
So a future where reality and history are subtly tweaked to the specifications of those willing to pay…
Google already scans your photos folder and offers enhancements, stitches together panoramas and so on. So inserting product placement is totally believable.
These scenarios were much talked about a decade back in relation to advertising on photographs on Facebook, specially with Coca Cola and other popular brands.
That would make a great Black Mirror episode... and a terrible dystopia if it becomes reality.
> Sensor quality in phones goes down, AI makes up for it
Why do you think the stock camera apps usually get better results?
How could a phone's smaller-than-a-thumbnail lens setup ever get as much light as a proper camera's?
Let's hope AI won't replace people faces with wrong ones.
https://www.theverge.com/21298762/face-depixelizer-ai-machin...
> This faked data, submitted by users as "real pics in real places" is further used to train AI models
I am begging people to find a new gotcha for AI other than "training it on data created by itself or other AI."
It's an obvious issue with obvious solutions. If it happens, it will be due to ignorance. It is not inevitable, and it shouldn't even be likely.
Why would we even need photos when we can hallucinate it all?
Can't grow cameras.
That isn't far from how iphones work now. They have mediocre cameras, people only think they are good because they throw a lot of AI image enhancement at it.
Is Apple’s AI adding hallucinated details? The last I read it’s just used to merge multiple images - up to 8 or 9 images - to form the final image. While I could see details getting lost or artifacts being added, I don’t think it can add actual “feature” details that don’t exist.
1 reply →
I would be more concerned with the impact on criminal system as more and more both defense and prosecution is dependent on cell phone camera data...
It is all faked by AI well.....
I wonder when the AI will hallucinate a gun into a black persons hand since the training black people often had guns? Hands moving fast are really blurry, so it has to hallucinate a lot, so it doesn't seem impossible. I could see that becoming a scandal of the century.
1 reply →
If (when?) we start replacing mirrors with cameras and a screen, it’s possible we may go through life entirely without knowing what we look like.
And everybody's body image improves as they only know the idealized version of themselves, but see the real version of their meatspace acquaintances.
In the other hand, there will likely be a market for makeup mirrors that emphasize your flaws (both to help the user and to sell more product)
It's already somewhat the case with mirrors as we always see the symmetry of our real image.
> hallucinated
<soapbox>confabulated</soapbox>
The novelty of things like instagram is wearing off. I see more people not bothering to pull out their phone. It's not just wanting to compete with the photos taken by narcissist on the internet, it's also just losing interest, and knowing things you share can and will be used against you.
This will be the next step in film “restorations” too.
A combination of ai models trained on high resolution textures and objects, models of the actors, and training from every frame of the movie that cal use the textures and geometry from multiple angles and cameras to “reconstruct” lost detail.
Reminds me of that CCC talk by David Kriesel https://youtu.be/zXXmhxbQ-hk "Don't trust any scan that you didn't falsify yourself"
A "smart" scanner tries to compress images.
Oh man... I thought you were going to, "the stop signs and strip malls were how we discovered there were aliens on the Moon (and Mars) that look exactly like us!".
Of course they would have perfect skin and expertly applied eye-liner and lipstick as well.
Quite the equivalent (to me) to many kids preferring the taste of "strawberry yoghurt" compared to real strawberries, because it's sweeter and has enhanced taste. Except for photos.
Not sure I like that future
I've seen this with CGI. CGI still looks awful somehow, but people about my age think it looks cinematic, and people a decade younger think it looks incredibly realistic.
Or, you know, Samsung sells ad placements in the enhanced images to do things like turn a can of Coke into a can of Pepsi, overwrite billboards in the background, etc.
If we aren't already living in a simulation, then we've begun a feedback loop so that in the near future, we very well may be.
There is going to be a nasty feedback loop of AI being trained on its own output that will probably cause improvement to plateau.
I think the limitations are in optics and the signal processing stack, rather than the CMOS sensor. A better lens can go a long way.
I made a post about exactly this feedback loop wrt ChatGPT and it got no attention outside of one comment dismissing my point.
This is actually making me want to start using film again because it is so much harder to fake.
Or better, the AI improves your shitty snapshots so they come out great. Every shot is beautifully framed, perfect composition, correct light balance, worthy of a master photographer. You can point your camera any old way at a thing and the resulting photo will be a masterpiece.
The details don't quite correspond to reality; to get the framing right the AI inserted a tree branch where there wasn't one, or moved that pillar to the left to get the composition lined up. But who care? Gorgeous photo, right?
And the thing is, I don't think anyone would care. You'd get the odd weird comparison where two people take a photo of the same place and it looks different for each of them. And you'd lose the ability to use the collected photos of humanity to map the world properly.
I think it's fascinating. Reality is what we remember it to be. We can have a better reality easily ;)
That could be fine as long as there is either a way to turn all that off (or better a way to selectively turn parts of it off) or a separate camera app available that lets you do that.
It's the future. Something hit your self-driving hover car and left a small dent. To get your insurance to pay for fixing the dent you have to send them a photo.
Your camera AI sees the dent as messing up the composition and removes it.
Your insurance company is Google Insurance (it's the future...Google ran out of social media and content delivery ideas to try for a while and suddenly abandon so they had to branch out to find new areas to try and then abandon). Google's insurance AI won't approve the claim because the photo shows no damage, and it is Google so you can't reach a human to help.
> Reality is what we remember it to be. We can have a better reality easily ;)
Cue Paris Syndrome, because expectations will also be of a better reality. Then you go somewhere, and eat something, and experience the mess that actually exists everywhere before some AI removed it from the record.
https://en.m.wikipedia.org/wiki/Paris_syndrome
The evidentiary value of photography will plummet.
"future?"
Something I learned long ago is that people typically don't want the truth, in general. They want fictional lies, they crave a false reality that makes them happy. Reality in and of itself, for most, is an utter drag if they're made constantly aware of it and dwell on it. When it comes to marketing, people eat up the propoganda techniques. They want to be fed this amazing thing even if it's not really all too amazing. They love that it tickles their reward center in the process.
This of course isn't always the case. When something is really important or significant people sometimes do want to know the truth as best they can. I want to know the car I'm purchasing isn't a lemon, I want to know the home I'm buying isn't a money pit, I want the doctor to to tell me if my health is good or bad (for some, under the condition the information is actionable), and so on.
When it comes to more frivolous things, for many, build the fantasy, sell them that farm to table meal you harvested from the dew drops this morning and hand cooked with the story of your suffering to Michelin star chef and how you're saving my local community by homing puppies from the local animal shelter with profits... even if you took something frozen, slapped it in the microwave and plated it and just donate $10 a month to your local animal shelter where you visited twice to create a pool of photos to market. For many, they want and crave the fantasy.
Progress made by science and tech has, for a brief fragment of history, established techniques and made practical, in some cases, to peel away all or at least some layers of fantasy away to reality. We started to pierce into cold hard reality and separate the signal of truth, as we can best understand it, from all the noise of ignorance and fantasy.
For many fantasy lovers, snakeoil salesmen, and con men, pulling away the veil of fantasy and noise has been a threat and there's been a consistent battle to undermine those efforts. The whole emergence and perpetuation of misinformation and recent "fake news" trends are just some of the latest popular approaches. We've been seeding our knowledge and information more recently with increasing degrees of falsehoods and pure fabrications.
Now, enter "AI," especially generative flavors. The same people who wanted to undermine truth are foaming at the mouth at the current ability to produce vast amounts of noise that in some cases are almost indistinguishable from reality from current techniques we have. Not only that, fantasy lovers en masse are excited at the new level of fantasy they can be sold. They really really don't care or want the truth. They really do just want "a good looking picture", "to make the summary interesting", or just see some neat picture. They don't care how accurate it is. Now people interested in the truth are facing a deluge of technologically enabled difficult to seperate noise production.
Is what I'm looking at close to reality? How many layers of noise are there I should consider when interpreting this piece of information? In the past, the layers used to be pretty managable, they were largely physical limitations or resource limitations to falsify the data to a point that couldn't be easily discerned. These days... it's becoming increasingly difficult to determine this and more and more information in various forms are leveraging more sophisticated and believable noise production. Technology has made this affordable to the masses and there are many parties with interest in setting the clock back to a world where the best story tellers are looked at as the oracles of modern time.
People often scoff at ChatGPT that it seeds or "hallucinates" to interpolate and extrapolate gaps of knowledge and make connections but it does so in a way that people like. It projects confidence, certainty, and in many cases it gives exactly what people want. To me, it's scary because it's providing a service the majority seem to want and creating an onslaught of noise that's more costly to debunk than it is to produce.
[dead]
I think they're probably right about the AI-sharpening using specific knowledge about the moon... However, they are wrong about the detail being gone in the gaussian-blurred image.
If they applied a perfect digital gaussian-blur, then that is reversible (except at the edges of the image, which are black in this case anyway). You still lose some detail due to rounding errors, but not nearly as much as you might expect.
A gaussian blur (and several other kinds of blur) are a convolution of the image with a specific blur function. A convolution is equivalent to simply multiplying pointwise the two functions in frequency space. As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
Author tested for this by doing the experiment again with detail clipped into highlights, completely gone, model detail was added back.
> To further drive home my point, I blurred the moon even further and clipped the highlights, which means the area which is above 216 in brightness gets clipped to pure white - there's no detail there, just a white blob - https://imgur.com/9XMgt06
> I zoomed in on the monitor showing that image and, guess what, again you see slapped on detail, even in the parts I explicitly clipped (made completely 100% white): https://imgur.com/9kichAp
While I think this is a great test, I'm not really sure what that second picture is supposed to be showing. Kinda seems like they used the wrong picture entirely.
2 replies →
I watched the video and in this case the "recovered" detail is clearly natural to me. The original case does look like some kind of moon-specific processing, but this one with clipped highlights seems natural and can be achieved using classical CV.
2 replies →
> As long as you know the blur function exactly, you can divide the final image by the gaussian function in frequency space and get the original image back (modulo rounding errors).
Those rounding errors are very important though. The Gaussian function goes to zero very quickly and dividing by small numbers is not a good idea.
If your deconvolving a noise free version of the original that also doesn't have any saturated pixels (in the black or white direction) then you can get the pretty close to the original back. I don't think this applies here because the OP is taking a picture of a screen that shows the blurred version, so we've got all kind of error sources. I think the OP is right: the camera is subbing in a known picture of the moon.
It would be interesting to see what happens with anisotropic blur for example, or with a picture of the moon with added fake details (words maybe?) and then blurred.
> However, they are wrong about the detail being gone in the gaussian-blurred image.
Well yes, but he also downsampled the image to 170x170. As far as I know, downsampled information is strictly lost, and unrecoverable without an external information source (like an AI model trained with pictures of moon).
I'm too lazy to downscale it myself, so here's a 180x180 picture of the moon from WP [1]. This looks about the same as the Samsung result [2]. They are not getting the original detail, but they are getting the detail they should expect if Samsung simply deconvolved the blurred image.
[1] https://upload.wikimedia.org/wikipedia/commons/thumb/2/2b/Lu...
[2] https://imgur.com/bXJOZgI
>If they applied a perfect digital gaussian-blur, then that is reversible
Not true. Deconvolution is a statistical estimate. Think about it. When you blur, colors get combined with their neighbors. Statistically this moves toward a middle grey. You're compressing the gamut of colors towards the middle, and thus losing information. Look at an extreme case - 2 pixels of mid-grey. It can be deconvoluted to itself, to a light and dark grey, or to one black and one white. All those deconvolutions are equally valid. There's no 1-to-1 inverse to a convolution. If you do a gaussian blur on a real photo and then a deconvolution algorithm you'll get a different image, with an arbitrary tuning, but probably biased towards max contrast in details and light noise, since that what people expect from such tools and what most real photos have. But, just like A.I. enhanced images, it's using statistics when filling in the missing data.
https://medium.com/@gonced8/can-you-recover-a-blurred-image-...
1 reply →
But the AI should not have learned to apply a Gaussian deconvolution kernel. If anything it should be applying a lens-based bokeh kernel instead. A true lens blur does not behave like a Gaussian blur.
They don't get an exact reconstruction of the original image. What happens if you apply Gaussian blur and then try to undo it with a bokeh kernel?
3 replies →
While the information might be recoverable, the information is not seen by the camera sensor. Hence I think the argument in the post stands. Some AI model/overlay magic is happening, pretending to display information the sensor simply did not receive.
You're forgetting it was also downscaled to 170x170, and later had the highlights clipped. Both are irreversible.
This is incorrect. The frequency domain inverse of the Gaussian ends up yielding a division by zero. There is no inverse for the Gaussian.
That is mathematically true but not practically. Though indeed the Gaussian kernel has lots of zeros [1], in actuality, (a) the zeros themselves are at points, not regions, and therefore of little consequence, and (b) in practice the noise generated from reamplifying frequencies near these zeros can be minimized via techniques such as Wiener deconvolution [2].
[1] https://en.wikipedia.org/wiki/Window_function#Gaussian_windo...
[2] https://en.wikipedia.org/wiki/Wiener_deconvolution
They didn't claim invertible. The de-gaussinization is a reversible process albeit not invertible. I actually say more in this comment
https://news.ycombinator.com/item?id=35111998
No amount of math is going to save the original detail from getting downsampled to 170x170
A major problem with blur beyond rounding errors, say due to the optics being somewhat blurry due to manufacturing difficulties and tradeoffs for weakening assembly tolerance requirements (like wanting rotationally symmetrical optical surfaces, despite a rectangular shaped actively-used image focal plane (e.g. CMOS photodiode array), and potential for specializing the design to evenly light up _just_ that rectangle), is that the photon shot noise has a standard deviation equal to the square root of the photon count.
A smartphone sensor pixel has space for some low 4 digits number of electrons (created with some probability from photons, but that stochastic effect doesn't matter for anything a normal user would photograph) and typically should have a fixed 2~10 electron standard deviation from the analog-to-digital-converter (well, mostly the amplifiers involved in that process).
So if your pixel is fully exposed at a high 10000 electrons, and you √ that, you have 100 electrons stddev from shot noise plus worst case 10 electrons stddev from the readout amplifier/ADC. If you have a dark pixel that only got 100x less light to only have accumulated 100 electrons, √ of that gives 10 electrons stddev of shot noise plus the same 10 electrons stddev readout amplifier/ADC.
The problem is that while you have an SNR of 5 with the dark pixel, when trying to deconvolve it out of a nearby bright pixel, even perfectly with no rounding errors (1 electron = 1 ulp/lsb in a linear raw format), you now have 100/110 = 10/11 ≈ 0.91. That's far worse than the 5 from before. This gets worse if your ADC has only the 2 electrons stddev instead of the 10 (about 2x worse here).
That's the reason why deconvolution after the photon detector is a band aid that you only begrudgingly tend to accept.
The trade-off just requires massively increased aperture/light gathering, likely negating your savings on optics.
> If they applied a perfect digital gaussian-blur, then that is reversible
Actually any noise distribution is frequently reversible if you know the parameters and number of steps. This is in fact how diffusion models work (there's even work of Normalizing Flows removing realistic camera noise). It is just almost impossible to figure this out since there are many equivalent looking ways. But we need to be clear that there is a difference between reversibility and invertibility. A invertible process is bijective, or perfectly recreates the original setting. A reversible process can just work in both directions and isn't guaranteed to be invertible. (Invertible means reversible but reversible doesn't mean invertible)[0]
I bring this up because even more complicated versions of bluring could be argued as not "faked" but rather "enhanced." A better way to test Samsung faking the data is to mask out regions. If the phone fills in the gaps then it is definitely generating new data. This can still be fine if the regions are small, unless we also want to call bilinear interpolation "faked" but I don't think most people would. This is why it gets quite difficult to actually prove Samsung is faking the image. I don't have a Samsung phone to test this though.
So basically I'm with you, and even a slightly stronger version of this
> It is not totally inconceivable that the AI model could have learned to do this deconvolution with the Gaussian blur function, in order to recover more detail from the image.
Edit: After reading other comments I wanted to bring some things up.
- The down scaling is reversible, but not invertible. We can upscale, reversing the process. But yes, there is information lost. But some data can still be approximated and/or inferred.
- The clipping experiment isn't that good. Honestly, looking at the two my brain fills in the pieces and they look reasonable to me too. Clipping the brightness isn't enough, especially since it is a small portion of the actual distribution. I did this on both the full image and small image and both are difficult to distinguish by eye from the non-clipped. Clipping below 200 seems to better wash out the bottom of the moon and remove that detail. 180 seems better though tbh.
The level of BS in this thread perfectly resembles the BS in religious-level audiophile discussions. A mixture of provably correct and provably incorrect statements all mixed together with common words used in uncommon ways.
> But yes, there is information lost. But some data can still be approximated and/or inferred.
The perfect summary.
2 replies →
I would be interested to see what the best possible deconvolution of the blurred image looks like, if anyone has the setup and knowledge to try it?
Gaussian blur is essentially acting as a low pass filter. An IR filter does not strictly destroy information in the filtered spectrum components, but does attenuate their power.
Given a perfect blurred image, reconstruction is possible - however due to the attenuation, these high frequency components are ~sensitive~.
Apart from quantisation effects [you mentioned which limits perfect de-convolution], adding a little AW Gaussian noise(such as taking a photo of the image from across the room) after the kernel is applied obliterates high frequency features.
Recovery when noise is low (plus known glyphs) is why you should not use Gaussian blur followed by print screen to redact documents. Inability to recover when there are artifacts and noise is [part of] why cameras cannot just set a fixed focus [at whatever distance] and deconvolve with the aperture [estimated width at each pixel] to deblur everything that was out of focus.
TLDR for readers, It is unlikely to recover sufficient detail via de-convolution here.
Is it Gaussian blur, though, or some other invertible kernel?
I wondered that too. I think he should have altered the moon image a bit before applying the filter.
Author did, did it again intentionally CLIPPING detail so there was none, not just blurred gone. It put detail.
1 reply →
This is wrong. The blurred image contains only intensity information, but reversing the convolution in frequency space would require phase information as well. A simple Gaussian blur is not reversible, even in principle.
There is no "phase information" in the spatial domain. "Phase" is literally, where the pixels are on the screen.
Rather, reversing blur of any type is limited (a) by spatial decimation (a.k.a. down sampling, which is performed in the article), and (b) by noise/quantization floor, below which high frequency content has been pushed.
3 replies →
Hey all, it's the author of the reddit post here. First of all, let me say that I don't usually frequent HN, but the comments on here are of such high quality, that I might need to change that. I got semi-depressed on reddit, with people misattributing statements and, in general, not being overly, uh, skeptical :)
That being said, there were a few comments on here about gaussian blur and deconvolution, which I would like to tackle. First, I need to mention that I do not have an maths/engineering background. I am familiar with some concepts, as I've used deconvolution via FFT several years ago during my PhD, but while I am aware of the process, I don't know all the details. I certainly didn't know that the image that was gaussian blurred could be sharpened perfectly - I will have to look into that. In fact, I used gaussian blur to redact some private information (like in screenshots), and it's very helpful to know if I haven't redacted anything and the data is recoverable. Wow.
I would love to learn more about the types of blur that cannot be deconvoluted.
However, please have in mind that in my experiment:
1) I also downsampled the image to 170x170, which, as far as I know, is an information-destructive process
2) The camera doesn't have the access to my original gaussian blurred image, but that image + whatever blur and distortion was introduced when I was taking the photo from far away, (whatever algo they are using doesn't have access to the original blurred image to run a perfect deconvolution on)
3) Lastly, I also clipped the highlights in the last example, which is also destructive (non-reversible), and the AI hallucinated details there as well
So I am comfortable saying that it's not deconvolution which "unblurs" the image and sharpens the details, but what I said - an AI model trained on moon images that uses image matching and a neural network to fill in the data.
Thank you again for your engagement and your thoughtful comments, I really appreciate them, and have learned a lot just by reading them!
> In fact, I used gaussian blur to redact some private information
Absolutely never do that. I honestly don't understand why people still do, given that it's obvious that low levels of blur can be reversed why even risk guessing until what point someone might be able to recover anything? Just censor it, draw over it with an opaque tool, and save it in a format that won't store layers or undo history or something (the riskiest format being pdf).
If you don't like how that looks, the alternative is to replace the information and then blur it. They can unblur but will find an easter egg at best.
Personally, I censor instead of blurring a replacement, but I balance between low contrast and not hiding the fact that information was removed. A stark contrast distracts and looks ugly. E.g., for black text on a white background, I'd pick a light/medium gray (around the average black level of the original text, basically).
> save it in a format that won't store layers or undo history or something
For eliminating such risk, just screenshot your censored content and use that image.
Should be rather easy to prove if Samsung is really able to „unblur“ an image in that way: use something else than an image of the moon as starting point and apply the same steps, i.e. down sizing and blur, then take a photo and see if it’s able to recover details.
I just wanted to say that the experiment at the end where you had half the moon and the whole moon was brilliant, and perfectly illustrated the problem in a single picture. If anyone hasn't seen that they should.
It would be interesting to see how this mode handles foreground objects such as an aeroplane or clouds.
Does it just overdraw it (i.e. erase it), apply texture over the foreground element, or fail altogether?
This scenario is probably why other vendors don't go so far as to fake such images with texture overlays.
I took a max-(non-optical)-zoom photo of a rabbit in my yard a while back using an iphone, then further enlarged the result to see how it did - in the details it looked like an impressionistic painting of a rabbit, facing the camera and looking left. The actual rabbit was looking away from the camera and to the right. The eye and face were not visible.
https://i.ibb.co/Kz7Sbm2/8-EA85-C12-5-B11-44-D8-9566-461-C98...
Look at pictures on social media in recent months, pretty much every single image looks like this. If you zoom in, they're all inaccurate impressionist paintings. They only look good at original size.
It's absolutely painful and encourages me yet again to use my mirrorless camera even more.
At this point I wish we had legislation requiring a "turn off AI bullshit" option for any camera. Every time I've taken a picture of myself using the camera app built-in to WhatsApp, it blurs the shit out of my face in an attempt to hide blemishes or whatever, and (in a weird sort of reverse-vanity) it really annoys me that it gives the impression I care about looking un-blemished, which I do not. AFAICT, there is no way to turn this off.
3 replies →
While I understand what the author tries to say, I have to point out that ship has long sailed. Samsung just pushed it a bit too far and slapped a "scene optimizer" label on it.
AI has been used in "cell phone photography" for a few years, at lease since Pixel 2 where a mediocre sensor produced much better pictures than what people expect (maybe there are other players who did this even earlier). And every manufacturer started doing it, including Apple. Otherwise, do you think "night mode" is just pure magic? Of course not, algorithms are used everywhere.
How do you define "fake"? In podcasts, Verge editor Nilay Patel has asked various people "what is a photo", because the concept of a "photo" has become increasingly blurry. That is the question the author is asking, and people may have different answers from the author's.
> Otherwise, do you think "night mode" is just pure magic?
Night mode definitely uses some AI but most of the result is from stacking frames. Samsung here did not label it as a "scene optimizer". Their marketing just calls it Space Zoom. The only disclaimer they provide is "Space Zoom includes digital zoom, which may cause some image deterioration."
According to Samsung -- and I just confirmed it on my S22 under Camera -> Camera Setting -- it's called "Scene Optimizer"[1]:
[1] https://r1-community-samsung-com.translate.goog/t5/camcyclop...
Note that this behaviour is limited to scene mode, which has a moon shoot mode. You can always use the normal or pro mode where the pictures are not magically enhanced.
Is is ridiculous that OP consider this "cheating". Most people just want a nice picture and don't give a damn about AI.
Photo is an interesting word. It's meaning is clarified by other words, such as photorealism, photofinish. These words will (strictly) lose their meaning if photograph simply means image captured and processed by a device.
Curiously and revealingly, the political word photo-op stands alone in this photo- parade of words in the age of photo-imaginings. The universe does indeed have a sense of humor.
Using algorithms to take multiple pictures and stack them together is fine. The information is real, exists, and objective. People in the background won't (for example) suddenly be facing the other way because of the algorithm.
The problem is that AI isn't just interpolating data. It is wholesale adding extra data that simply doesn't exist. The person in the background is facing left, but the sensor couldn't possibly have captured that detail even after multiple images--it was a coin flip that the AI made.
The issue is that, like privacy, most people won't care ... until they do. By that time, it will be too late.
> People in the background won't (for example) suddenly be facing the other way because of the algorithm.
Someone here included an example where it does do something like this: https://news.ycombinator.com/item?id=35109568
The software technology in the original pixel cameras were using multiple frames of varying exposure to allow for impressive dynamic range in images while still retaining colour and contrast. This is quite a difficult thing to do as requires precise understanding of what the 'edge' of an object is, and I think that is what AI was used for. This stacking technique is also used for night exposures.
I'm sure that they have started using AI to fill in details more recently, but this is just to point out clever use of multiple exposures and AI can help without faking detail.
eg "Do We See Through a Microscope?" https://philpapers.org/archive/HACDWS.pdf
> the concept of a "photo" has become increasingly blurry
nice.
This problem is not new. In 2019, Huawei introduced a special image processing feature in its smartphone camera app, the "Moon Mode" (opt-in). Missing details are added to the moon photos via machine learning inference from a pre-trained model. Huawei then started marketing these processed images as a showcase of its new smartphone's photography performance. In China, it was widely criticized by tech reviewers [1][2] as misleading, and "Moon Mode" became a running gag among tech enthusiasts for a while.
It seems that Samsung simply adopted the same tactic to compete...
On the Huawei "Moon Mode" controversy, one can even find a research paper [3] published in a peer-reviewed social studies (!) journal, Media, Culture & Society:
> This is where the controversy began: Chinese tech critic Wang’s (2019) posting on Weibo, the Chinese equivalent of Twitter, made quite a splash. In his post, Wang put forward a shocking argument: he said that Huawei’s Moon Mode actually photoshops moon images. He contended that, based on his self-conducted experiments, the system ‘paints in pre-existing imagery’ onto photographed takes, re-constructing details that are not captured in the original shots. Huawei immediately refuted these claims, stressing that the Moon Mode system ‘operates on the same principle as other Master AI modes that recognize and optimize details within an image to help individuals take better photo'
[1] https://www.androidauthority.com/huawei-p30-pro-moon-mode-co...
[2] https://www.phonearena.com/news/Is-the-Moon-Mode-on-the-Huaw...
[3] https://journals.sagepub.com/doi/full/10.1177/01634437211064...
This couldn't be the same Huawei who has been caught repeatedly using DSLR stock photography in their marketing materials, while claiming the images were taken by their smartphones.
https://petapixel.com/2020/04/21/huawei-accidentally-claims-...
It would be so fun to hook a phone up to a telescope and take a picture of, say, Jupiter, and see if it overlays the ringed planet with the moon's characteristics.
My hypothesis is that the neural network was trained on a lot of labeled photos, so somewhere inside the network, when you see the moon, it has some moon=0.95 confidence number, and whatever label has the highest confidence, it tries to bring it up to 1.0 akin to how deepdream makes images of spaghetti have more dog faces. Samsungs marketing department interprets that as technically enhancement of images and not faking the moon specifically. So perhaps if it sees Jupiter, it will try to make it more jupitery.
I knew I saw this before.
This is just make hipsters get into old point and shoot digital cameras... good thing I kept my Canon A540.
All the subtle trickery manipulation that the smart phone's doing to reality is concerning. Smoothing people's faces, making their eyes pop, enhancing the shit out of the colours, and now plopping fake objects overtop of the real ones.
Future concerns of this technology should range from a low-key disconnect from reality, to the complete inability to photograph certain objects or locations.
Imagine dusting off a 30 year old digital camera, finding some AA batteries to put in it, snap a selfie and then realizing just how ugly we all are and how washed out the polluted world actually looks without a bunch of narcissism-pandering enhancements.
Not sure about hipsters, but it's apparently somewhat of a trend with young people.
"The Hottest Gen Z Gadget Is a 20-Year-Old Digital Camera
Young people are opting for point-and-shoots and blurry photos."
https://www.nytimes.com/2023/01/07/technology/digital-camera...
They are realizing that these things are all toys and fashion so you might as well save money and just buy old stuff and above all take photos.
I think it's great. It's the exact opposite of the person who spends all their time gear shopping and never using the gear.
6 replies →
I'm 40 and did the same thing, bought a cheap analog camera with black and white film to shoot fun photos on my birthday party :) The film is still at the lab, but I guess it's worth the wait.
I love my Canon AE-1 but dang it's expensive to actually use.
The G9x Mark II, Sony RX100, Panasonic LUMIX and similar 1" sensor cameras are awesome though and I don't think they've gotten too crazy with computational photography. I imagine some color processing modes might be doing a bit of work though.
> Smoothing people's faces, making their eyes pop, enhancing the shit out of the colours,
Our brains do far worse stuff with our memories. Not sure how relevant 'high fidelity' is to people who mostly use phones for memories
That’s what photos used to be good for. They don’t fudge stuff like our brains.
3 replies →
I regularly take photos of text etc because I am not going to remember it. If a photo of a config password is AI fucked into showing the wrong digits, there’s a real problem.
“Future concerns of this technology should range from a low-key disconnect from reality…”
This phrase could cast a broadening net with each year’s new tech.
> I downsized it to 170x170 pixels and applied a gaussian blur, so that all the detail is GONE. This means it's not recoverable, the information is just not there, it's digitally blurred
Strictly speaking, applying a Gaussian blur does not destroy the information. You can undo a Gaussian blur with a simple deconvolution, which is something I would expect even a non-AI image enhancement algorithm to do (given that, you know, lenses are involved here).
I'd like to see what detail can be "recovered" with just the downsizing, which DOES destroy information.
Well the op did downsize so details had to be reconstructed. Also the noise from having the image being projected through a screen and then a retaken through the camera sensor means that it isn't just your standard perfect convolution.
They say they also clipped all whites above a certain level too. That’s just information that’s been destroyed and then invented by the AI right?
if you downsize an image to 170x170px and then blow it up so it's visible to a camera from across the room without any sort of blurring, it's not going to look like anything and the camera's object detection won't recognize it as the moon - it's just going to look like a huge pixel grid.
I am not quite so confident. I would like to see an experiment to test how badly you can distort an image of the moon before the AI stops recognising it.
1 reply →
OP here again.
I photoshopped one moon next to another (to see if one moon would get the AI treatment, while another would not), and managed to coax the AI to do exactly that.
This is the image that I used, which contains 2 blurred moons: https://imgur.com/kMv1XAx
I replicated my original setup, shot the monitor from across the room, and got this: https://imgur.com/RSHAz1l
As you can see, one moon got the "AI enhancement", while the other one shows what was actually visible to the sensor - a blurry mess
I think this settles it.
Famously, upscaled Obama: https://twitter.com/Chicken3gg/status/1274314622447820801
I picture someone 20 years from now trying to find out what their parent really looked like when they were young. The obviously smoothed-out face filters are already giving way to AI-powered homogenization. And the filtering is moving deeper down the stack from the app to the camera itself. There will be no "original".
This issue has been known for a few years. Here's a more thorough analysis from January 2021:
"Is the Galaxy S21 Ultra using AI to fake detailed Moon photos?": https://www.inverse.com/input/reviews/is-samsung-galaxy-s21-...
Note that the author decides that Samsung's photos are "not fake", in the sense that they were not doctored after being taken with the phone. However, the article decisively proves that they're being heavily doctored in-camera.
Another test would be to shoot RAW + JPEG if the camera supports it. A true RAW image would reveal what the sensor is actually capturing.
Depends. For example Apple can save ProRaw format which DOES include the postprocess magickery, but uncompressed and having more detail for post.
BUT, it also provides apps with true RAW, which is true (or very close to true) raw.
Two different definitions of "raw" is apparently possible even on the same device.
> For example Apple can save ProRaw format which DOES include the postprocess magickery, but uncompressed and having more detail for post. BUT, it also provides apps with true RAW, which is true (or very close to true) raw.
Thank you for that detail. IMO, that makes "ProRAW" a really unfortunate naming decision on Apple's part. And I think you're saying that if you shoot in RAW with, say, Halide, the result will be an actual RAW (pre-demosaic) file in DNG format.
A couple of interesting articles on the topic: (1) https://lux.camera/understanding-proraw/ (2) https://www.austinmann.com/trek/iphone-proraw
Pentax K3-II 300mm: https://cdn.discordapp.com/attachments/1010562706237038633/1...
Sensor: 23.5 x 15.6mm 24MP
S21 Ultra: https://cdn.discordapp.com/attachments/1010562706237038633/1...
Telephoto sensor: 3.3 x 4.3mm 10MP (240mm equivalent)
Compare side by side: https://imgur.com/a/QwnV99D
I have a similar photo done on S22 ultra on March 6 this year, and neither look like your (position of lower right mega crater but also the rest). So its not simple 'photoshop-into-predefined-nice-image'.
I can clearly see that most folks here don't actually own discussed devices (which is fine, its US-based HN, a bastion of iphone and many Apple employees dwell here and uncritical appreciation of Apple is very evident in every single related thread). I've used its 10x zoom extensively over more than a year, it simply blows all other phones away easily for that kind of situation (more than those rather weak 3x zooms available everywhere). Family photos, wild animals, nature, anything you want to come closer, otherwise the scene is tiny dots in the center like on other phones. It works really well for what it is, with obvious unavoidable physical limits.
Overall this phone made me put my fullframe Nikon D750 away on a day I bought it. I took it 'just to be sure' on vacation to Egypt last year, didn't touch it a single time. Most often it doesn't produce strictly as good images but a) they are good enough to be viewed on phones side by side easily, basically as good as fullframe there and sometimes even much better, ie handheld photos in the night of dark scenes, fullframe is utterly lost without tripod, and b) it weights 0 and takes 0 extra space (and cost 0 instead of many thousands for modern camera with big sensor), since I have phone with me always anyway.
Tried exactly the steps as author of article, couldn't reproduce it a bit, tried various mega zooms, his various original photos, dark room etc. Blur remained blur, nothing added. I mean at this point everybody acknowledges any decent phone is painting quite a bit (ie iphone taking other side of bunny than reality, thats a fine example) and I am sure Samsung is doing their part as they have the literal android flagships.
To be clear, are you saying the difference in orientation was introduced by the camera, or is there something more subtle going on here?
Orientation is because photos were taken months apart.
This is to get an idea of quality, with the APS-C camera having a much larger (26x area) sensor and better optics to work with.
My impression is... S21 Ultra "space zoom" is, at best, a good party trick. But if you zoom in, the quality is still nearly garbage. Not an objectively "great photo of the moon."
2 replies →
I'm not very convinced. It looks like they are doing some processing, maybe special to the moon but it looks more like some form of sharpening or contrast boosting than adding detail. In all of the examples it seems that there is information in the original (dark spots) that are getting boosted.
It would be interesting to see this tried on a source image that isn't the moon. Just white with a few dark spots. Does it actually add in completely new craters, or just where there are existing smudges? Or do something like half of a moon photo and have white, does it add craters to the white side?
The OP tried to do this by changing the contrast but I failed to see any craters appearing where there wasn't already dark spots in the source photo.
It does seem strange that the OP is using an image of the moon to start and that they don't provide a still shot of the one where they modified the brightness levels to cause clipping. It doesn't really "drive the point home" as claimed.
Of course the answer to these may be that you need something moon-like enough to trigger the moon optimizations. But if that is the answer it would be interesting to see something that comes right up to the threshold where it either snaps in and out of these optimizations or two very similar images produce widely different results.
> In all of the examples it seems that there is information in the original (dark spots) that are getting boosted.
That was the point of the Gaussian blur. By blurring the source image of the moon before taking its picture, there was no information. The Gaussian blur destroys fine detail, by design.
The image enhancement applied by the Samsung phone is adding detail where there was none originally – not just detail that might be buried in the optics somewhere, but not there at all. It involves a computational model of what the target (here the moon) "should" look like and guesses at what it thinks are the blurry bits.
No, Gaussian blur does not "destroy" fine detail. It is still there it is just that the "volume" of it is turned down. You can put all the fine detail back again by applying the inverse filter.
Gaussian blur is reversible with deconvolution - however that is almost certainly not what is happening here, as it’s fairly computationally expensive.
3 replies →
But I don't see it adding any detail over the blur. It really just looks like it is boosting contrast a bit on top of the blur. I don't see anything like ridges or defined features appearing.
Does it enhance the moon only, or the Taj Mahal, Grand Canyon and Montblanc, too?
How long until your phone detects the subject based on geo localization and replaces your shot with a stock image of the subject from an immense database of selected, professional-looking pictures?
Someone made a camera that has this "feature": https://philippschmitt.com/archive/2018/work/camera-restrict...
This doesn't replace your photo, it restricts you from taking it entirely
Officially documented by Samsung in Korean:
https://r1.community.samsung.com/t5/camcyclopedia/%EB%8B%AC-...
Ongoing arguments over what this counts as:
https://old.reddit.com/r/Android/comments/11or39c/samsungs_a...
https://twitter.com/search?q=samsung%20moon
DeepL.com translation of relevant bit (there's some irony in using ML to be able to read text about ML):
> However, the lunar photography environment has physical limitations due to the long distance to the moon and lack of light, so the actual image output from the sensor at high magnification is noisy and not enough to provide the best image quality experience even after compositing multiple images.
> To overcome this problem, Galaxy Camera applies an AI detail enhancement engine (Detail Enhancement technology) based on deep learning at the final stage to effectively remove noise and maximize the details of the moon, resulting in bright and clear moon photos.
It's not just this one thing, though. It also describes moon detection resulting in: setting brightness, using optical stabilization, "motion sensor data and video analysis"¹ for stabilization (VDIS), and fixing the focus at infinity. The "AI" magic is then described as being a detail enhancer, which could mean anything from {generically decreasing blur and improving contrast} to {applying a model that was purposefully overfit on the moon} (most likely it's the latter). Either way it's fake, obviously, but why do all the effort and make compromises to get a good moon shot if you're going to replace it anyway?
> If you shoot the moon in the early evening, the sky around the moon will not be the color of the sky you see, but rather a black sky, which is caused by forcing the image brightness down to capture the moon clearly. [I think this translation should have read "picture brightness"; elsewhere it also says "screen brightness" so I suspect the Korean word for "picture" is ambiguous]
So you can't shoot anything near the moon, like if someone is holding the moon up with a hand or something, presumably that would be all black. It's apparently still relying on the sensor to get most of the way there and using ML for the last leg.
Imo the feature should spawn a warning on screen "Details of moon filled in by computer and may differ from reality" with buttons for [ok] to quickly dismiss as well as [don't show again]. Then you can't not know that your images are being faked and it's not disingenuous, while most people would still appreciate the better quality because it'll be/work fine in 99% of cases.
¹ summary explanation of VDIS found on https://r2.community.samsung.com/t5/CamCyclopedia/VDIS-Video...
I guess there is some AI algorithm that does zoom as postprocessing. That AI knows the moon so it can fill in the blanks and compensate for a (relatively) crappy sensor.
So in the future, there are either cameras that can see what others have seen before, and those that can truly capture new, true, detail (true as in, without filling it with estimations)
Samsung is learning the best from Huawei https://www.androidauthority.com/huawei-p30-pro-moon-mode-co...
Idea for further testing to prove the hypothesis beyond reasonable doubt:
What happens if you manually add a few additional artificial details to your copy of the image -- like a trap street in maps. Does the camera slip up and show flawless moon instead?
Or if you skew image a bit, or do a photo of blurred image of another planet, or any absolutly any other black and white blurred photo?
It enhances everything. This reddit user doesn't know what modern photography on a smartphone entails.
I have a few questions: Is this specific for the moon or is the AI generally sharpening around dark areas? If it is specific to the moon then how many common objects does the camera recognize? If you did the same test with a picture of a stop sign or a popular car would the results be similar? Also is this processing done on the camera or does this require an internet connection and work on Samsungs end? I have known about AI image enhancement for years but the idea of recognizing then re-texturing common objects is something I had never considered.
Can it be just deconvolution? https://en.wikipedia.org/wiki/Deconvolution
The Gaussian blur they applied is theoretically reversible in a continuous function and infinite precision. In a discrete function (like a image in the computer) and only a few dozens of bits it's not 100% reversible, but it can be partially undone and get a sharper image (that is not as sharp as the initial image).
No. The author downsampled the image to 170x170 pixels and clipped the white levels so that details were turned to uniform white.
I guess one way to tell whether the image is processed locally is by disabling every connection, and seeing if the pics still look good.
But there still might be advanced machine learning models, rather than simple filters.
I guess the old "seeing is believing" can be thrown out the window nowadays.
With photographs it was never true. This is the classical text on it.
https://en.wikipedia.org/wiki/The_Work_of_Art_in_the_Age_of_...
But it isn't seeing, right? I mean, seeing a digital image of a thing is totally different from seeing the actual thing.
A digital image is a representation/abstraction/map kind of thing. And then a little bit of mental shorthand inside your head makes a connection and declares the thing and the representation-of-the-thing to be one and the same.
But it isn't the same thing. Not by a million miles.
It's a socially-accepted mindfuck is what it is.
Oh but the plot thickens. Was this post itself a result of AI plagiarizing/paraphrasing of another older post?
https://old.reddit.com/r/Android/comments/11nzrb0/samsung_sp...
It's creepy how deceptive phone cameras are. I've noticed in pictures that I've taken that certain elements have been postprocessed to look better than they actually are.
Samsung is such a tasteless company.
This is a cool and useful tech, but it obviously needs a good marketing story to avoid looking creepy. To think that no one will notice just makes it worse.
yeah 5 minutes with their flavor of android makes that apparent, woof
A gaussian blur is not a good test. It does not technically remove all the information from the image. As such this test can't distinguish between unblur or actual moon-pasting.
Binning the image, or cropping it in Fourier space, would be a better test.
They also downscaled it to 170x170 (Which is basically binning) and clipped the highlights at 216 (Which is basically Fourier space cropping)
I think it’s even more damning seeing this happen on the phone near the bottom of the article. There’s no way a phone camera sensor can capture all those raw values of the moon with 45X digital zoom with a phone telephoto lens (you can only see so far with 5 mm of packaging). From a digital sensor point of view, it probably sees a white blob and fires up the AI when it has a contrast and lighting similar to the moon
What would be even funnier (and more obvious, once people caught on) would be if the phone used its orientation sensors to only activate "Moon Mode" when pointed at the approximate current location of the moon. :)
Wait ... a Gaussian blur? That doesn't remove ANY information from the image. It looks blurred to our eyes but the same information is all there.
The information is absolutely not gone.
Does it state that it hallucinates craters that were not there in the original, or is it possible the filters simply did an FFT, adjusted the power spectrum to what we expect of a non-blurry picture, hence inverting the Gaussian blur?
EDIT: Note that a deblur of a smooth but "noisy" image can cause "simulation" or "hallucination" entirely without AI. Could be any number of things causing an output image like that (wavelet sharpening, power spectrum calibration, ...). Even if the information isn't recoverable as such, a photo of a Gaussian blur has an unnatural power spectrum that could easily "trick" conventional non-AI algorithms into doing such things.
Especially since the only thing I see in the output is "more detail" (i.e. simply a different power spectrum than the author expected..)
Unless you’re using floating point numbers and have padded the image to infinity, it’s literally untrue that applying a Gaussian kernel doesn’t throw away at least some (if not a lot of) information. I think what you intended to say is the blur doesn’t remove “all” the information. Not to mention he downsized the image (likely using bilinear not even bicubic ) further throwing out a ton of information. Not to mention he then showed the image on a display and took a photo from across the room. For all practical purposes the technically incorrect pedantic statement does not apply even if you correct it.
Looking at the images, it will be fantastical if not a violation of information theory if the phone didn’t use prior information about how the moon looks to create the photo.
Source: spent way more months than I would have preferred calculating CRLB values on Gaussian blurred point source images.
Hear Hear! This deserves to be reiterated given the statements of information recoverability made in this thread.
It also happened after downsampling (aka removing information) of the image. And when parts of the image was clipped to white (aka no information at all). So your point doesn't absolve Samsung of guilt as you believe.
Yeah the author glossed over this a bit in my opinion. In infinite precision math you're correct, but at some point the signal in those higher frequencies is going to be reduced below the precision of the storage data type, never mind the dynamic range of the monitor and camera he's using.
It seems clear the author does not have knowledge on the subject, more than glossing over -- the article even emphasises that information is removed by the blur (100% wrong). I agree it may not have destroyed the experiment entirely, but it does mean the experiment was conducted without knowledge of basic signal processing and I would prefer a more through study or analysis before drawing conclusions...
1 reply →
> The information is absolutely not gone.
Continue to read through paragraphs 4 and 5 of the "Conclusion" section.
Well it is a conclusion written without knowledge of signal analysis.
The input to the algorithm will be an image with a power spectrum that isn't natural, . It would be very natural even without "AI" to attempt to "deblur" when faced with such a power spectrum.
A deblur of noise can cause "hallucinations" with much simpler reasons than AI beong involved.
Could it not e.g. be doing some wavelet transforms or FFTs and automatic power spectrum calibration?
1 reply →
How it is possible that people belived that these images are not artificially enchanced? Like it is obvious, there is no way a phone sensor can take such detailed photos of the Moon. I am surprised this was even a discussion.
How am I supposed to know how much detail a phone's sensor can capture?
The most impressive thing is how dumb the Reddit comments are. Incorrect factual claim after incorrect factual claim. Confidently stated and widely upvoted.
Was it always like this? Were we, people always like this?
Have you read comments here? Confidently exclaiming that downscaling, gaussian blur, and taking a picture of screen showing that from far away can easily be restored by deconvolution...
> Confidently exclaiming that downscaling, gaussian blur, and taking a picture of screen showing that from far away can easily be restored by deconvolution...
I don't think HN (or Reddit) posters are arguing that deconvolution was actually being used, or that it would be practical to do so. Instead, they're just nitpicking about the technical correctness of this particular statement - "the (ir)reversibility of a perfect digital gaussian blur" - even if it doesn't affect the conclusion of the experiment.
It's just the standard nerd-sniping as seen in all tech communities... If the debate on whether you can "reverse a photo of a gaussian blur taken by a camera from the monitor" became hot enough, eventually someone may even spend a weekend to code a prototype to show how it actually works better than most people's imagination, with the solely purpose to win an argument on the Internet.
2 replies →
Hah! Those were some of the comments on the original Reddit thread I was commenting on. Glad to see that confident ignorance is alive here too.
Nearly every forum is like that. It may even be related to Dunning Kruger, in that people who overestimate their knowledge are obviously going to feel knowledgeable on more things.
That said, Reddit is probably the worst place for it. I rarely read comments anymore there because of how 'dumb' they are. Worse, I feel like Reddit is like some self trained botnet at times. Go on any post, say 'blood is thicker than water', and without fail someone is going to tell you the original is 'blood of covenant is thicker than water of the womb.' This is of course incorrect, and the only place I've ever seen it is...Reddit. So people posting ridiculous things learning from people posting ridiculous things.
It happens here though, too, with the main difference being someone will usually correct you, and not be downvoted for doing so...
I am not convinced. Looks like a bog standard sharpening alghorithm to me.
I’ve taken pictures of the moon at 100x optical zoom, and if Samsung is really faking this they’re doing a truly awful job of it.
MP3 was perceptually-aware lossy compression. How many people complain about it 20 years later
First - I see that you don't have any audiophile friends! Many sound geeks dislike the sound of compressed audio.
Second - there's nothing wrong with an AI 'enhancement' option. It seems useful! But you should admit what you are doing and have an off switch. Imagine if all of the studio masters for the past 20 years were actually in lossy mp3 because the hardware switched silently?
> First - I see that you don't have any audiophile friends! Many sound geeks dislike the sound of compressed audio.
Don't let them tell you this unless they can pass an ABX test.
They dislike the fact that playing lossy audio
The changes the AI makes are perceptually noticeable though when compared to source.
No one complains about image or video compression either if the quality is good enough - because it’s not perceptually noticeable vs uncompressed; people can do A/B testing to be sure.
MP3 doesn't claim it's lossless. Samsung claims they're not using a moon stamp.
I tinkered with the idea to let a app use your personal gallery as training data and create a "barney camera"(himym) where 100% of photos of you and your friends would always be perfect, it would be impossible to capture a bad photo.. same here, its the moon just perfect .. who cares its not like it changes ..
Blatant but how is this different than all of the other "AI" touchups to things like faces?
Samsung adds craters to the moon. Imagine your phone added other features to your face.
What happens if you film a crime in the apartment far away and the AI fills in the details?
This question is less hypothetical than it seems. In recent years, there has been amazing breakthroughs in low-light photography with machine learning, such as [1]. It's not hard to imagine that it would be applied to future surveillance cameras.
I imagine that in most cases, the "AI" would work just fine at guessing things. But important edge cases exist, including the possibility of adversarial machine learning spoofing attacks to create false images. Imagine faking a crime scene this way, it would something straight from cyberpunk fiction.
[1] Learning to See in the Dark https://cchen156.github.io/SID.html
Samsung is a corrupt company with a corrupt culture.
Starting from the top - bribery, embezzlement, illegal transactions, stock manipulation, perjury - all the way down to their "partners".
They've been caught multiple times lying in their products description.
I won't add links as there are too many, but a quick search for Samsung and the keywords I mentioned will bring many results.
(let the downvoting begin)
So apparently iPhone and Pixel phones are not adding non existent details to Moon unlike Samsung
https://www.reddit.com/r/Android/comments/11onztx/uibreakpho...
In my experience the iPhone camera is basically incapable of taking moon photos at all. You just get a 100% white circle.
Does DxOMark penalize such falsifications?
Who is DxOMark?
They are a organization that perform benchmarks. Here's a link: https://www.dxomark.com/smartphones/
1 reply →
It’s camera testing software.
Every few months there's a new clown on reddit that doesn't know anything about photography or gaussian blur and tries to expose smartphones for using AI based sharpening and interpolation.
Oh nooooo this phones photos come out much sharper than my peanut brain expected!!! Fraud!!!!!!!!!!
He didn't just blur it, he downsized it first
> I downsized it to 170x170 pixels and applied a gaussian blur, so that all the detail is GONE.
Has anyone found a decent use of the 100x zoom on these phones? Could it be that the sheer hype it causes compensates for the extra cost of putting it on the phone? Like, even if nobody uses it it stills turn an overall profit because of the marketing
Not sure about the current situation, but a couple years ago could receive tons of updoots on Reddit for posting a video how you zoom eg from one side of the harbour to see something on the other side.
In this regard, the AI is acting just like the human brain - adding details that may or not be present in what the viewer is seeing. Details that the viewer expects to see, so the brain delivers against this expectation.
Samsung Galaxy Phone - now hallucinating a more beautiful reality™
Maybe I can get a selfie with my S23 and get a picture of Leo DiCaprio!
Huawei phones did this a few years back.
You can get sued if you publish someones real pictures without makeup and photo-shopping. AI beatified pictures have the same criteria.
> You can get sued if you publish someones real pictures without makeup and photo-shopping.
That sounds really weird. Any further info?
we need "raw" camera apps for our smartfones. like oraganic food, "no AI was used in the making of this photo"
Do we really need cameras at this point? Remove cameras, and just ask user what he is taking photo of, then generate the image.
Still need some cheap camera sensor so that you have data to upload to Facebook and the FBI, though. Otherwise, I've hypothesized the same.
Am I supposed to be convinced by this? Cause I'm not.
It looks more like moon mode is assembling fake detail out of shot noise from a photo being taken in the dark indoors. That's enough to cancel out his blurring and all that.
Even if it is guided upsampling with a "this is a moon prior", so what? That's not "fake", it's constrained by the picture you took.
Wasn’t this obvious from Day 1
It seems somebody from VW started working at Samsung
That's not a moon, it's a space station.
I'm surprised that some people are surprised.
tl;dr: AI is used to add detail that doesn't exist to your photos.
[dead]
[dead]
So this person has been able to replicate a process that he thinks Samsung must've used, and this is proof? What a load of rubbish this article is!
Not denying that Samsung or any other brand would fake things, but this is in no way any proof at all.
I think you misread the article. What they did was:
1. Use Photoshop to blur a photo of the moon, destroying detail
2. Use a Samsung camera to take a photo of the blurred photo of the moon
3. The camera somehow emits a crisp photo of the moon, including the detail that was destroyed in step 1
It seems like the camera AI detected 'this is a photo of the moon', and used its knowledge of what the moon looks like to add the detail back in. Where else could the camera have gotten the detail from?
Ah yes, now I get it! I either didnt read the article clearly or you outline the steps much better than the article does! My bad.