Comment by supriyo-biswas
1 month ago
The referenced Apple blog post[1] is pretty clear on what this feature does, and I wish the author at lapcatsoftware (as well as folks here) would have read it too, instead of taking the blog post as-is.
Apple has implemented homomorphic encryption[2], which they can use to compute distance metrics such as cosine similarity without revealing the original query/embedding to the server. In the case of photos, an on-device model is first used to detect if a landmark may be possibly present in the image, and then sends an encrypted embedding[3] for the image region containing said landmark, which is evaluated on the server using HE techniques and then the encrypted results are sent back.
I'm sure someone will come along and say Apple does none of these things; in which case, said commenter should probably not use Apple devices since, there is no reason to trust the toggles for "Enhanced Visual Search", and perhaps more importantly, the one for "Advanced Data Protection" either. However, I rarely see any other major company researching ML and HE together, so that alone is a sufficient positive indicator in my eyes.
(It's interesting that this is also being downvoted. Please go ahead, I can't stop you :P but please also write the parts that you disagree with. Thank you.)
[1] https://machinelearning.apple.com/research/homomorphic-encry...
[2] https://en.wikipedia.org/wiki/Homomorphic_encryption
[3] https://mlr.cdn-apple.com/video/HE_Fig_3_PNNS_889f3a279c.m4v
> please also write the parts that you disagree with. Thank you
The problem invoked by the article is that data is being sent back to Apple by default. Saying "it's fine because it's encrypted" and "don't use apple if you're not fine with that" doesn't help.
The post complains about a product that stored sensitive customer content locally now sends that data to Apple, and given the combination of abuse on privacy and horrendous, generalized security failures that we've seen across the industry, those concerns seem genuine. Your comment is very dismissive of these concerns, which would explain why it's being down voted.
Apple also makes a mandatory API call to captive.apple.com from every device with a web view, just about every time they connect to a network.
If someone is willing to take a hardline stance that a vendor should inform the user and require opt-in consent for every type of exchange, they are going to have to run platforms specifically targeted toward that mindset.
You're extensively describing the technical implementation while missing the fundamental issue: Why is Apple enabling this feature by default for what is, essentially, a luxury photo search feature?
Let's break this down:
1. Nobody is questioning whether Apple's technical implementation is sophisticated or secure. It clearly is.
2. Nobody is suggesting the privacy toggles don't work. They do.
3. The issue is about Apple deciding that automatically sending data about users' photos (regardless of how securely) is an acceptable default for a feature many users may never want or need.
Consider the value proposition here: Apple invested significant engineering resources into complex homomorphic encryption and differential privacy... so users can search for landmarks in their photos? And they deemed this feature so essential that it should be enabled by default?
This feels like using a golden vault with military-grade security to store grocery lists. Yes, the security is impressive, but that's not the point. The point is: Why are my grocery lists being moved to the vault without asking me first?
A privacy-respecting approach would be: "Hey, would you like to enable enhanced landmark search in your photos? We've built some really impressive privacy protections to make this secure..."
Instead of: "We've already started analyzing your photos for landmarks because we built really impressive privacy protections..."
The sophistication of the technology doesn't justify making it the default for what is ultimately an optional convenience feature.
> users may never want or need
Are you assuming that Apple did not perform a market analysis when implementing this feature? I think that is unlikely, considering the effort.
Thank you for posting the apple blog post. As usual, it's really interesting research, and it's fascinating to see how they solve potentially gnarly privacy issues.
> Thank you for posting the apple blog post.
The very same blog post is linked in the submitted article.
Everything about what they did was absolutely fantastic and amazing, until they turned it on by default.
That's not the point of the outrage though (at least not for me). They enabled by default a feature that analyzes my pictures (which I never upload to iCloud) and sends information about them to their (and others') servers. That is a gross violation of privacy.
To be clear, I don't care about any encryption scheme they may be using, the gist is that they feel entitled to reach into their users' most private data (the photos they explicitly said they don't want to upload to iCloud) and "analyze" them.
This is the same as that time Microsoft enabled OneDrive "by mistake" and started slurping people's private documents and photos saved in default locations (arguably worse since no one takes pictures with their PC's webcams).
If you really didn't want your photos to be analyzed, would you be using an iPhone? Or any modern smartphone? Google photos doesn't have nearly the privacy focus and no HE whatsoever but I rarely see that mentioned here. It almost seems like Apple gets held to a higher standard just because they have privacy preserving initiatives. Do you use a keyboard on your iphone? You may not have heard but apple is tracking which emojis you type most often [0] and they get sent to apple servers.
[0] https://www.apple.com/privacy/docs/Differential_Privacy_Over...
> Google photos doesn't have nearly the privacy focus and no HE whatsoever but I rarely see that mentioned here. It almost seems like Apple gets held to a higher standard just because they have privacy preserving initiatives.
What is so surprising about this? If you make big claims about anything, you are held to your own standards.
> It almost seems like Apple gets held to a higher standard just because they have privacy preserving initiatives
It doesnt' seem this way at all. It is rare to see someone talking about current behavior, they are always talking about the slippery slope - such as landmark detection obviously being the first step in detecting propaganda on a political dissident's phone.
This isn't driven by trying to hold them to a higher standard; it is an emotional desire of wanting to see them caught in a lie.
Like parent mentioned - I don't upload photos to Google photos, assume parent doesn't upload photos to iCloud.
Should photo info be sent to Apple/Google in this case?
If the data is encrypted, does the concern still apply?
You bring up the example of Onedrive, but there is no use of e2e encryption or HE techniques there.
> does the concern still apply?
Yes it does and the blogpost specifically explains why.
In short, both iOS and macOS are full of bugs, often with the potential of exposing sensitive information.
Also, it’s on by default - nobody in their sane mind would have bits of their photos uploaded somewhere, regardless of “we promise we won’t look”.
Finally, Photos in iOS 18 is such a bad experience that it seems the breach of privacy was fundamentally unjustified as no meaningful improvement was introduced at all.
7 replies →
Yes, of course, the concern is the data being exfiltrated to begin with. Like someone else in this thread mentioned, if they upload a single pixel from my image without my consent, that is too much data being uploaded without my consent.
3 replies →
> If the data is encrypted, does the concern still apply?
Yes! For so many reasons!
If an adversary is able to intercept encrypted communications, they can store it in hopes of decrypting it in the future in the event that a feasible attack against the cryptosystem emerges. I don't know how likely this is to happen against homomorphic encryption schemes, but the answer is not zero.
I'm not suggesting everyone should spend time worrying about cryptosystems being cracked all day long, and I'm not saying Apple's encryption scheme here will prove insecure. Even if this particular scheme is cracked, it's very possible it won't reveal much of great interest anyways, and again, that is simply not the point.
The point is that the correct way to guarantee that your data is private is to simply never transmit it or any metadata related to it over a network in any form. This definitely limits what you can do, but it's a completely achievable goal: before smartphones, and on early smartphones, this was the default behavior of taking pictures with any digital camera, and it's pretty upsetting that it's becoming incredibly hard to the point of being nearly impractical to get modern devices to behave this way and not just fling data around all over the place willy-nilly.
And I know people would like Apple to get credit for at least attempting to make their features plausibly-private, but I feel like it's just the wrong thing right now. What we need today is software that gives agency back to the user, and the first part of that is not sending data off to the network without some form of user intent, without dark patterns to coerce said intent. At best, I can say that I hope Apple's approach to cloud services becomes the new baseline for cloud services, but in my opinion, it's not the future of privacy. The future of privacy is turning the fucking radio off. Why the fuck should we all buy mobile devices with $1000 worth of cutting edge hardware just to offload all of the hard compute problems to a cloud server?
I'd also like to ask a different question: if there's no reason to ever worry about this feature, then why is there even an option to turn it off in the first place?
I worry that what Apple is really doing with pushing out all these fancy features, including their maligned CSAM scanning initiative, is trying to get ahead of regulations and position themselves as the baseline standard. In that future, there's a possibility that options to turn off features like these will disappear.
5 replies →
I thought less about the data, as much as the metadata, direct and indirect.
Just from memory when the scheme came up in earlier discussion.
The system is essentially scanning for the signature for some known set of images of abuse so that it aims to capture abusers who would naively keep just these images on their machines. (It can't determine if a new image is abusive, notably).
It's conceivable some number of (foolish and abusive) people will be caught this way and those favoring a long dragnet for this stuff will be happy. But this opens the possibility that a hacker could upload an image to an innocent person's computer and get that person arrested. Those favoring the long dragnet will naturally say the ends justify the means and you can't make an omelet without cracking a few eggs. Oh, "think of the children".
Edit: Also worth adding that once a company is scanning user content to try to decide if the user is bad, it makes it that much easier to scan all kind of content in all kind of ways for all kinds of reasons. "for the good", naturally.
That was the CSAM thing that they have announced they gave up on.
This is totally different.
Are you sure? Publicly yeah, but the same technology can easily be used for the same purpose.
2 replies →
>The system is essentially scanning for the signature for some known set of images of abuse
Huh? The system is scanning for landmarks, not images of abuse.
>people will be caught this way
Due to the homomorphic encryption, I don't think Apple even knows whether the image matches a landmark in Apple's server database or not. So even if Apple put some images of abuse into its server database (which Apple claims only contains pictures of landmarks), I don't think Apple would know whether there was a match or not.
Does Apple explicitly say that?
Or only that they don’t know which landmark it matched?
1 reply →