What I want is very simple: I want software that doesn't send anything to the Internet without some explicit intent first. All of that work to try to make this feature plausibly private is cool engineering work, and there's absolutely nothing wrong with implementing a feature like this, but it should absolutely be opt-in.
Trust in software will continue to erode until software stops treating end users and their data and resources (e.g. network connections) as the vendor's own playground. Local on-device data shouldn't be leaking out of radio interfaces unexpectedly, period. There should be a user intent tied to any feature where local data is sent out to the network.
So why didn't Apple just simply ask for user permission to enable this feature? My cynical opinion is because Apple knows some portion of users would instantly disallow this if prompted, but they feel they know better than those users. I don't like this attitude, and I suspect it is the same reason why there is an increasing discontent growing towards opt-out telemetry, too.
This mindset is how we got those awful cookie banners.
Even more dialogs that most users will blindly tap "Allow" to will not fix the problem.
Society has collectively decided (spiritually) that it is ok signing over data access rights to third parties. Adding friction to this punishes 98% of people in service of the 2% who aren't going to use these services anyway.
Sure, a more educated populous might tip the scales. But it's not reality, and the best UX reflects reality.
Nope, collective indifference to subpar user experiences has gotten us those lousy cookie banners.
Web sites could legally use cookies for non-tracking purposes without cookie banners but considering people have not stopped visiting sites despite the fugly click-through cookie banners makes them a failure.
All it takes is for 50% of the internet users to stop visiting web sites with them, and web site authors will stop tracking users with external cookies.
No. A significant number of people care about Privacy which is why 1. Apply was targeting them with Ads and 2. AdBlock did hurt Google's business. Also care is different from go to war (as in install Linux and manually setup a privacy shield + Tor + only transact in Monero). Some people do that out of principal. Many people want the Privacy features but with the ease of use.
> This mindset is how we got those awful cookie banners.
The only thing I've found awful is the mindset of the people implementing the banners.
That you feel frustration over that every company has a cookie banner, is exactly the goal. The companies could decide that it isn't worth frustrating the user over something trivial like website analytics, as they could get that without having to show a cookie banner at all.
But no, they want all the data, even though they most likely don't use all of it, and therefore are forced to show the cookie banner.
Then you as a user see that banner, and instead of thinking "What a shitty company that don't even do the minimal work to not having to show me the cookie banner", you end up thinking "What a bad law forcing the company to inform me about what they do with my data". Sounds so backwards, but you're not the first with this sentiment, so the PR departments of the companies seems like they've succeed in re-pointing the blame...
I think society has collectively "decided" in the same way they "decided" smoking in a restaurant is great.
There's little to no conscious choice in this. But there is a lot of money in this. Like... a LOT of money. If I were to try to influence society to be okay with it, it would be a no brainer.
So, to me, it's obvious that society has been brainwashed and propagandized to accept it. But doing so generates hundreds of billions if not trillions of dollars. How, exactly, such manipulation is done is unknown to me. Probably meticulously, over the course of decades if not centuries. I know that the concept of privacy during the writing of the constitution was much, much more stringent than it was in the 70s, which is much more stringent than it is today.
I think it's clear that users should be able to have their own agents that make these decisions. If you want an agent that always defers to you and asks about Internet access, great. If you want one that accepts it all great. If you want one that uses some fancy logic, great.
u-Block Origin's annoyances filters take care of the cookie banners, giving the best of both worlds: no banners and a minimal amount of tracking.
(The "I don't care about cookies" extension is similarly effective, but since I'm already running u-block origin, it makes more sense to me to enable it's filter.)
Users had a global way to signal “do not track me” in their browser. I don’t know why regulators didn’t mandate respecting that instead of cookie consent popups.
Apple IDs could easily have global settings about what you are comfortable with, and then have their apps respect them.
I’m spitballing here but wouldn’t another way to handle it would be to return dummy / null responses by redirecting telemetry calls to something that will do so?
This would have the added benefit of being configurable and work on a bunch of apps instead of just one at a time too
Not really. A mandatory opt-in option at the browser level would be the correct way to do it, but legislation forced instead those cookie banners onto the webpage.
The vast majority (>95%) of users does not understand what those pop-ups say, seems fundamentally incapable of reading them, and either always accepts, always rejects, or always clicks the more visually-appealing button.
Try observing a family member who is not in tech and not in the professional managerial class, and ask them what pop-up they just dismissed and why. It's one of the best lessons in the interactions between tech and privacy you can get.
Well, then >95% of users won't be using $FEATURE. Simple as that. The fact that users for some reason no not consent to $FEATURE the way corporations/shareholders would want them to does not give anyone the right to stop asking for consent in the first place.
When looked at from another angle, opt-in does work.
By adding that extra step forcing users to be aware of (and optionally decline) the vendors collection of personal data, it adds a disincentive for collecting the data in the first place.
In other words, opt-in can be thought of as a way to encourage vendors to change their behaviour. Consumers who don't see an opt-in will eventually know that the vendor isn't collecting their information compared to others and trust the product more.
As much as I hate cookie consent dialogs everywhere, the fact is that it is clearly working. Some companies are going as far as to force users to pay money in order to be able to opt out of data collection. If it wasn't so cumbersome to opt-out, I reckon the numbers for opt-out would be even higher. And if companies weren't so concerned about the small portion of users that opt-out, they wouldn't have invested in finding so many different dark patterns to make it hard.
It is definitely true that most users don't know what they're opting out of, they just understand that they have basically nothing to gain anyway, so why opt-in?
But actually, that's totally fine and working as intended. To be fair to the end user, Apple has done something extremely complicated here, and it's going to be extremely hard for anyone except for an expert to understand it. A privacy-conscious user could make the best call by just opting out of any of these features. An everyday user might simply choose to not opt-in because they don't really care about the feature in the first place: I suspect that's the real reason why many people opt-out in the first place, you don't need to understand privacy risks to know you don't give a shit about the feature anyway.
If you do not want it (and that is >90% of people, who never asked for it, never requested it, but was forced upon them these 'enriched' lies and exposure to corporate greed).
> Try observing a family member who is not in tech
This is everyone, it is universal, I've met many people "in tech" who also click the most "visually appealing" button because they are trying to dismiss everything in their way to get to the action they are trying to complete.
The microcosm that is HN users might not just dismiss things at the 95%+ rate, but that is because we are fed, every day, how our data is being misappropriated ate every level. I think outside of these tiny communities, even people in tech, are just clicking the pretty button and making the dialog go away.
The issue really isn't opt-in itself but how the option is presented.
I agree that a lot of people don't read, or attempt to understand the UI being presented to them in any meaningful manner. It really is frustrating seeing that happen.
But, think about the "colorful" option you briefly mentioned. Dark patterns have promoted this kind of behaviour from popups. The whole interaction pattern has been forever tainted. You need to present it in another way.
Informed consent is sexy. In the Apple ecosystem, we’re literally paying customers. This is ridiculous. This line you parroted is ridiculous. This needs to stop.
> So why didn't Apple just simply ask for user permission to enable this feature?
That’s an interesting question. Something to consider, iOS photos has allowed you to search for photos using the address the photo was taken at. To do that requires the Photos app to take the lat/long of a photos location, and do a reverse-geo lookup to get a human understandable address. Something that pretty much always involves querying a global reverse-geo service.
Do you consider this feature to be a violation of your privacy, requiring an opt-in? If not, then how is a reverse-geo lookup service more private than a landmark lookup service?
It's a complete violation if it's a new or changed setting from the default state of the user not having it possible.
Something to consider - location is geo-encoded already into photos and doesn't need this uploaded to Apple servers. Searching can be done locally on device for location.
Apple goes as far as to offer a setting to allow the user to share photos and remove the geocoding from it.
Offering a new feature is opt-in.
Unfortunately, against my better wishes, this only erodes trust and confidence in Apple that if this is happening visibly, what could be happening that is unknown.
> Do you consider this feature to be a violation of your privacy, requiring an opt-in?
I suppose in some sense it is, as it a reverse-geo lookup service, but it's also no where near to the front in the location privacy war.
Cell phone providers basically know your exact position at all times when you have your phone on you, credit card companies know basically everything, cars track driving directly, etc. etc.
I can see why some people would be up in arms but for me this one doesn't feel like missing the forest for the trees, it feels like missing the forest for the leaves.
You’re right. But: Anyone in IT or tech, thinking deeply about the raw facts. They know it always boils down to trust, not technology.
The interesting thing is that Apple has created a cathedral of seemingly objective sexy technical details that feel like security. But since it’s all trust, feelings matter!
So my answer is, if it feels like a privacy violation, it is. Your technical comparison will be more persuasive if you presented it in Computer Modern in a white paper, or if you are an important Substack author or reply guy, or maybe take a cue from the shawarma guy on Valencia Street and do a hunger strike while comparing two ways to get location info.
Personally I do not believe these popups serve any purpose, because I ultimately cannot (at least in a reasonable way) prove that the website is acting in good faith. Asking me whether the app should phone home doesn't really guarantee me pressing "no" will actually prevent the tracking.
I am continuously surprised at how we convince ourselves privacy at scale will work with a varying amount of yes/no buttons. There are 2 ways to trust software 1. be naive and check whether "privacy first" is written somewhere 2. understand the software you are running, down to the instructions it is able to execute.
The permission popups also lack granularity. When giving access to my contact list, which contacts does it actually access? Can I only give access to contacts name and not phone numbers? Is it for offline or online processing? If online, should we have another popup for internet access? But then, can I filter what kind of internet stuff it does? You go down the rabbit hole and eventually end up with a turing-complete permission system, and if you don't, your "privacy" will have some hole to it.
Even with opt-in a vendor will keep harassing the user until they tap "yes" in an inattentive moment.
And I've been in situations where I noticed a box was checked that I'm sure I didn't check. I want to turn these things off and throw away the key. But of course the vendor will never allow me to. Therefore I use Linux.
For what it's worth, I use Linux, too, but as far as phones go, stock phones that run Linux suffer from too many reliability and stability issues for me to daily drive them. I actually did try. So, as far as phones go, I'm stuck with the Android/iOS duopoly like anyone else.
> I want software that doesn't send anything to the Internet without some explicit intent first
I want this too, but when even the two most popular base OSes don't adhere to this, I feel like it's an impossible uphill battle to want the software running on those platforms to behave like that.
"Local-first" just isn't in their vocabulary or best-interest, considering the environment they act in today, sadly.
Developers of software want, and feel entitled to, the data on your computer, both about your usage within the app, as well as things you do outside of the app (such as where you go and what you buy).
Software will continue to spy on people so long as it is not technically prohibited or banned.
I highly suggest everyone else does their darnedest not too either. Don’t do it in your own software. Refuse and push back against it at $dayJob.
I realize that my small contribution as a privacy and data-respecting SWE is extremely small, but if we all push back against the MBAs telling us to do these things, the world will be better off.
In the OP article it seems more like users demand to search their photos by text, and Apple has put in a huge effort to enable that without gaining access to your photos.
Years ago I developed for iOS as an employee. In my case, it was the product managers that wanted the data. I saw it as a pattern and I hated it. I made my plans to leave that space.
> So why didn't Apple just simply ask for user permission to enable this feature? My cynical opinion is because Apple knows some portion of users would instantly disallow this if prompted, but they feel they know better than those users. I don't like this attitude, and I suspect it is the same reason why there is an increasing discontent growing towards opt-out telemetry, too.
I'm just not sure why Apple needed to activate this by default, other than not draw attention to it... and doing so that was more important than the user's rights to the privacy they believe they are purchasing on their device.
I don't care what convenience i'm being offered or sold. If the user has decided what they want and the premium they are paying for Apple, it must be respected.
This makes me wonder if there is an app that can monitor all settings in an iPhone both for changes between updates, and also new features being set by default to be enabled that compromise the user's known wishes.
Consent for complex issues is a cop out for addressing privacy concerns. Users will accept or reject these things without any understanding of what they are doing either way. Apple seems to have taken a middle ground where they de-risked the process and made it a default.
This is a “look at me, Apple bad” story that harvests attention. It sets the premise that this is an unknown and undocumented process, then proceeds to explain it from Apple documentation and published papers.
"What I want is very simple: I want software that doesn't send anything to the Internet without some explicit intent first."
It exists. I use such software everyday. For example, I am submitting this comment using a text-only browser that does not auto-load resources.
But this type of smaller, simpler software is not popular.
For example, everyone commenting in this thread is likely using a browser that auto-loads resources to submit their comments. HN is more or less a text-only website and this "feature" is not technically necessary for submitting comments. All so-called "modern" web browsers send requests to the internet without explicit intent first. IN addition to auto-loading resources, these browsers automatically run Javascript which often sends further requests never intended by the web user.
Brand new Apple computers now send packets to the internet as soon as the owner plugs them in for the first time. This may enable tracking and/or data collection. Apple proponents would likely argue "convenience" is the goal. This might be true. But the goal is not the issue. The issue is how much the computer owner is allowed to control the computer they buy. Some owners might prefer that the computer should not automatically send packets to remote Apple servers. Often it is not even possible to disable this behaviour. Computer purchasers never asked for these "convenenience" features. Like the subject of this submission, Apple Photos, these are Apple's decisions. The computer owner is not allowed to make decisions about whether to enable or disable "convenience" features.
As the court acknowledged in its opinion in US v Google, default settings are significant. In this case, it is more than a default setting. It is something the owner cannot change.
>I want software that doesn't send anything to the Internet without some explicit intent first.
I too want exactly that, which got me thinking, that's what firewalls are for! DROP OUTBOUND by default, explicit allow per-app.
On Andoid, iptables-based firewalls require root, which wasn't a good option for me (no twrp support for my device), so after some searching I stumbled upon NetGuard - open source and rootless, implements a firewall using Android's VPN service (you can configure Android to route all traffic through this "VPN" which is actually a local firewall). The downside is you can't use an actual VPN (except with some complicated setup involving work profiles and other apps). I've been using it for a couple of weeks and am very satisfied, I noticed apps phoning home which I did not want to, like a scanning app I had used to scan private documents in the past, perhaps an oversight on my part.
Use a rooted Android phone with AFWall+ installed, with default block rules. Even just LineageOS allows you to set granular network settings per app, though it's not preemptive like AFWall.
Can't run various banking apps and can't run PagerDuty on a rooted device due to Google Play API Integrity Check. The ecosystem is closing in on any options to not send telemetry, and Google is leading the way in the restrictions on Freedom.
Absolutely! The important bit is that users have no choice in the matter. They're pushed into agreeing to whatever ToS and updating to whatever software version.
The backlash against Microsoft's Windows Recall should serve as a good indicator of just how deeply people have grown to distrust tech companies. But Microsoft can keep turning the screws, and don't you know it, a couple years from now everyone will be running Windows 11 anyways.
It's the same for Android. If you really want your Android phone to be truly private, you can root it and flash a custom ROM with microG and an application firewall. Sounds good! And now you've lost access to banking apps, NFC payments, games, and a myriad of other things, because your device no longer passes SafetyNet checks. You can play a cat-and-mouse game with breaking said checks, but the clock is ticking, as remote attestation will remove what remains of your agency as soon as possible. And all of that for a notably worse experience with less features and more problems.
(Sidenote: I think banking apps requiring SafetyNet passing is the dumbest thing on planet earth. You guys know I can just sign into the website with my mobile browser anyways, right? You aren't winning anything here.)
But most users are never going to do that. Most users will boot into their stock ROM, where data is siphoned by default and you have to agree to more data siphoning to use basic features. Every year, users will continue to give up every last bit of agency and privacy so as long as tech companies are allowed to continue to take it.
One of the reasons is because telemetry and backdoors are invisible. If the phone was showing a message like "sending your data to Cupertino" then users were better aware of this. Sadly I doubt there will be a legal requirement to do this.
Apple seems to be the best option here too. They seem to have put in a huge effort to provide features people demand (searching by landmarks in this case) without having to share your private data.
It would have been so much easier for them to just send the whole photo as is to a server and process it remotely like Google does.
Whether or not people in general are aware of this issue and care about it, I think it's pretty disingenuous to characterize people as willfully giving up their privacy because they own smartphone. When stuff like this is happening on both iOS and Android, it's not feasible to avoid this without just opting out of having a smartphone entirely, and representing as a binary choice of "choose privacy or choose not to care about privacy" is counterproductive, condescending, and a huge oversimplification.
Do you honestly believe people understand what they’re doing?
Nowhere in marketing materials or what passes for documentation on iOS we see an explanation of the risks and what it means for one’s identity to be sold off to data brokers. It’s all “our 950 partners to enhance your experience” bs.
The shorter answer is that it's your data, but it's their service. If you want privacy, you should use your own service.
And for how cheap and trivial syncing photos is, any mandatory or exclusive integration of services between app/platform/device vendors needs to be scrutinized heavily by the FTC.
> Trust in software will continue to erode until software stops treating end users and their data and resources (e.g. network connections) as the vendor's own playground. Local on-device data shouldn't be leaking out of radio interfaces unexpectedly, period. There should be a user intent tied to any feature where local data is sent out to the network.
I find that there is a specific niche group of people who care very much about these things. But the rest of the world doesn't. They don't want to care about all these little settings they're just "Oh cool it knows it's the Eiffel tower". The only people who are becoming distrusting of software are a specific niche group of people and I highly suspect they're going to be mad about something.
> So why didn't Apple just simply ask for user permission to enable this feature?
Because most people don't even care to look at the new features for a software update. And let's be serious that includes most of us here otherwise, this feature would have been obvious. So why create a feature that no one will use? It doesn't make sense. So you enable it for everyone and those who don't want it opt-out.
I want a hardware mic switch. We are an iHouse with one exception and that's a SheildTV that is currently out of order because I want to reset it and haven't found time in, oh..., weeks. Anyway, out of the blue one of the kids asked about Turkish delights and wonders where the name came from. SO and I facepalm then explain. Not an hour later she gets something in her Facebook feed: 15 interesting facts about Turkey.
This is just too much of a coincidence. I know, I know, this "... isn't Apple's fault" blah blah. Bullshit it's not. They can't have it both ways where they say their app store process is great and then they allow this shit.
Browsing the Internet is explicit intent! Some of the stuff enabled by JavaScript definitely tows the line but at the very least that's not really the direct fault of the browser.
Would you mind giving an example of something bad that could happen to somebody as a result of Apple sending this data to itself? Something concrete, where the harm would be realized, for example somebody being hurt physically, emotionally, psychologically, economically, etc
Once upon a time, I worked for a pretty big company (fortune 500ish) and had access to production data. When a colleague didn't show up at work as they were expected, I looked up their location in our tracking database. They were in the wrong country -- but I can't finish this story here.
Needless to say, if an Apple employee wanted to stalk someone (say an abusive partner, creep, whatever), the fact that this stuff phones home means that the employee can deduce where they are located. I've heard stories from the early days of Facebook about employees reading partner's Facebook messages, back before they took that kind of stuff seriously.
People work at these places, and not all people are good.
Easy, consider a parent taking pictures of their kid's genitals to send to their doctor to investigate a medical condition, the pictures getting flagged and reported to the authorities as being child pornography by an automated enforcement algorithm, leading to a 10-month criminal investigation of the parent.
This exact thing happened with Google's algorithm using AI to hunt for CP[1], so it isn't hard to imagine that it could happen with Apple software, too.
and there's absolutely nothing wrong with implementing a feature like this, but it should absolutely be opt-in
This feature is intended to spy on the user. Those kinds of features can't be opt-in. (And yeah, holomorophic "privacy preserving" encryption song-and-dance, I read about that when it came out, etc).
This is an incredibly shallow dismissal that states the opposite of Apple's claim with zero evidence or reasoning and hand-waves away the very real and well-researched field of homomorphic encryption.
Users of my (free, open-source) app seem surprised to learn that we've got zero insight into usage patterns. There are situations where a small amount of anonymous telemetry would be extremely helpful but I'm not going to touch it with a barge-pole.
Opt-in makes the data useless - not just in terms of the huge drop in quantity but because of the fact it introduces a huge bias in the data selected - the people that would opt-in are probably not a good sample of "typical users".
Opt-out - no matter what safeguards or assurances I could provide is unacceptable to a subset of users and they will forcefully communicate this to you.
Don't get me wrong - I understand both the ease at which bad actors abuse telemetry and the ease in which "anonymous data" can prove to be nothing of the kind in a multitude of surprising ways.
But it's hard not to feel a little sad in a "this is why we can't have nice things" kind of way.
I can't remember where I saw this before. However, there was a site that collected analytics data client side in a circular buffer (or something), and there was a menu in the settings to send it back one-time or always, or download it yourself. If you experienced an error, they would pop up in a toast to share the analytics data with them so they could help fix the problem. You could, of course, decline.
That was probably the best system I'd seen, but I can't remember what site it was.
Consent is the key issue binding all. There is complete lack of consent when there is no opt-out and great degradation when the default is opt-out. Trust is the only means to consent.
1) Opt-in, Opt-survey, Opt-out is the only ternary to build trust. Survey is an active validator of trust and assists in low-bandwith communication. Question should be presented to the end user the first time using it or the next time the application starts and this feature was added.
2) Provide the exact analytical information you want to the end user so they can parse it too. The means to self-evaluate allowed information to be shared with providing the reports or views improves trust.
3) Known privilege to trust leads to more consent. Having priority support with features and bugs could be aligned with those that Opt-in. Analytical history / performance may assisting in solving the recent bug that was reporter.
Apple, Microsoft, Google, and all apply ambiguity to their analytical sharing without details, not how they use it and can abuse it. Most don't even provide an Opt-out. I don't trust these organizations but I must engage with them through my life. I don't have to use Facebook or Twitter and don't. I accept the Steam survey.
RFC with an agreed upon analytical standard could be step to solving the latch of analytical information the open source community would benefit from. Both parties consenting to agreed upon communication.
*My Point of View; meta data is still personal data. Without the user the data and the meta data would not existing. Since the end user is the entropy to meta data they own the meta and the data.
Would there be a way to do the stats gathering on device, then once every few months send a popup with statistics?
Not sure what bias it adds
Like
"hey, we make this app, and we care about privacy, here is the information we have gathered over your usage for the past month, can we send this to ourselves, so that we can use it to improve the app?"
And then show human readable form of what data was collected.
Just as a reference of existing implementations of this: This is essentially how Valve/Steam collects hardware details from users/clients. Every now and then, a popup appears asking the user if they'd like to participate in the "Hardware Survey", together with all the data that would be submitted if they accept.
The podcast app I use, AntennaPod (far better for me than other apps, available on F-Droid, no affiliation!) just gave me a local-only year in review. I thought it was a great touch, and would be happy to have then shared the data from that with the app's makers.
You'd still have extremely biased data - people who blindly click OK on every pop up are not representative of your typical user; people who get nightmares after hearing the word "telemetry" and will gather the pitchforks if they hear any hint of will always refuse, but depending on your app, might be your typical user (e.g. for self-hosted picture sync and catalogue, who is the target audience - people who don't trust Apple/Google/Amazon/Dropbox to store their images privately)
This sort of sounds like the Steam Hardware Survey. They do not collect the data willy-nilly, they ask you every few months if you want to participate in a one-time check.
I have an incentive to see if the Linux desktop share has increased, so I usually run the survey for them to get my data point in. I also suppose the "gamer" crowed likes to show off how powerful their "rig" is, so I would imagine they commonly also run the survey for that reason as well.
> Opt-in makes the data useless - not just in terms of the huge drop in quantity but because of the fact it introduces a huge bias in the data selected - the people that would opt-in are probably not a good sample of "typical users".
Why? I don't think that's obvious. It may also be related to the way the opt-in is presented. In general, I would expect this to be a workable solution. Even if the opt-in group deviates from the "typical user", it's the best data you can get in an honest and ethically sound way. This should certainly be better than no data at all?
For any website/app that presents an opt-in cookie consent banner this is implicitly already the case.
Yes, this is one of the main reasons people mostly build on web. It's very difficult to make desktop software better, and especially Linux users are hostile to patterns that would make improvements possible
Hardly. It just has some issues with regards to what you also pointed out, bias for one. But it still provides valuable insight into usage patterns, systemic issues, and enables tracking effects of developments over time. Correcting the bias is not a bigger task than it is now - I'm sure you already have an idea about feedback to different features according to reviews, user reports, discussions, and so on. Opt-in is the same, just much better.
Maybe the solution lies in radical transparency: explaining exactly how and why telemetry would help, then letting users decide. But even that requires trust...
Is there a Github API for creating issues? I also maintain a free, open-source app and would love to make it easy for a crash to give users a button that opens a Github issues form--allowing users to see what crash data is populated and submit it if they want.
Data collection and telemetry is sadly lemon market type of situation. The most trustworthy developers are precisely the ones who don't collect data from users
This can only ever be opt-in if you want to stay on the legal side of the GDPR (and equivalents in other jurisdictions). You can ask, but the default needs to be "no" if no answer is given.
I provide telemetry data to KDE, because they default to collecting none, and KDE is an open-source and transparent project that I'd like to help if I can. If I used your app, I would be likely to click yes, since it's open-source. Part of the problem I have with projects collecting user data is the dark patterns used or the illegal opt-out mechanism, which will make me decline sending telemetry every time, or even make me ditch it for an alternative. An app that asks:
Can we collect some anonymized data in order to improve the app?
[Yes] [No]
...with equal weight given to both options, is much more likely to have me click Yes if none of the buttons are big and blue whilst the other choice is in a smaller font and "tucked away" underneath the other (or worse, in a corner or hidden behind a sub-menu).
Plus, I would think that SOME data would be better than NO data, even if there's an inherent bias leaning towards privacy-minded/power users.
> This can only ever be opt-in if you want to stay on the legal side of the GDPR
The GDPR only applies to personal data. You can collect things like performance data without opt-in (or even an opt-out option) as long as you are careful to not collect any data that can be used to identify an individual, so no unique device IDs or anything like that. Of course, you should be transparent about what you collect. You also have to be careful about combinations of data points that may be innocuous on their own but can be used to identify a person when combined with other data points.
> the only way to guarantee computing privacy is to not send data off the device.
> It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations. [...] By enabling the "feature" without asking, Apple disrespects users and their preferences. I never wanted my iPhone to phone home to Apple.
Regardless of how obfuscated or "secure" or otherwise "privacy-protecting" the feature is, the fact is that some information derived from one's personal content is transmitted, without prior consent. Even if the information is protected, all network queries are information. A timestamp that proves you took a certain action at a certain time (like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo), from a certain location (by correlating your location information at that time), etc etc.. and that's just the tip of the iceberg. Transmitting information from a user's device without their explicit consent is a violation of their privacy.
So Signal messages aren't secure because they're transmitted and so their "obfuscation" isn't enough to protect your data? Have you read what the author cited (and then admitted to not understanding) what Apple says they actually do to the data before transmission?
I could see an argument in the metadata (though there are multiple assumptions involved there, not least that they don't truly do OHTTP but instead conspire to learn at what timestamp a user took a picture), but if you already don't trust in what is essentially math, I'm not sure where the uncertainty and doubt ends
The difference being that the signal message is sent with consent: You literally press a button to send it there is a clear causal relationship between clicking the button and the message being sent.
The obvious difference is that by sending your photos with Signal, you are doing it willingly. You let it encrypt and decrypt willingly. You decide who gets it.
> like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo
So you jumped to a conclusion based on an incorrect premise. This is easy to see that this does not happen immediately after taking a photo. One the network traffic will show this (and it won’t), two homomorphic encryption is expensive so it cannot. Photos classically doesn’t sync on demand, as most iPhone users will know by way if it telling you this in the photos app when it does sync. Most expensive operations are queued up for when the device is plugged in (and on WiFi) because it’ll otherwise drain battery.
you're splitting a very fine hair while ignoring the larger privacy implication of the feature. So the timestamp might or might not be delayed a bit from being perfectly accurate? So what? It still is data approximating when the photo was taken, even if the resolution were as bad as "within a few days"
How much size would it take to store a model of every known location in the world and common things?
For ex: I sent a friend a photo of my puppy in the bathtub and her Airpods (via iphone) announced "(name) sent you a photo of a dog in a bathtub". She thought it was really cool and so do I personally. That's a useful feature. IDK how much that requires going off-device though.
I’m not an expert, but I would say extremely small.
For comparison Hunyuan video encodes a shit-ton of videos and rudimentary real world physics understanding, at very high quality in only 13B parameters.
LLAMA 3.3 encodes a good chunk of all the knowledge available to humanity in only 70B parameters.
And this is only considering open source models, the closed source one may be even more efficient.
These issues are all addressed in the Apple blog post that talks about how this feature is implemented. Two steps are taken to deal with these risks:
1) iOS creates additional fake queries, and all queries pass through scheduler that ensures you can use time-of-lookup to either discriminate real queries from fake queries, or identify when a photo was taken.
2) All queries are performed anonymously, with the use of a third party relaying service. So there’s no way for Apple to tie a specific query back to a specific device, or even IP address.
Between those two mitigating features. Getting hold of an individuals personal data using this feature requires you to first compromise the targets phone, to disable the fake queries. Then compromise the relaying party to correlate queries back to a specific IP address.
If you can manage all that, then quite frankly you’re a fool for expending all that effort. When you could just use your iOS compromise to have the device send you its location data directly. No need to faff about waiting for your target to take photos, then track multiple landmark lookups, carefully collecting a few bits of additional data per query, until you finally have enough to identify the location of your target or targets.
Is there a way to verify the claims of obfuscation, security and privacy? Or is the only verifiable fact the sending of unknown data to apple by the photos app?
My thoughts exactly: "we've got this crafty image fingerprinting, the CSAM detection use proved too controversial to roll out, but let's get the core flows into something that sounds useful for users, so the code atays alive, improving, & ready for future expansion."
Whether such fingerprinting can reliably be limited to public "landmarks" is an interesting question, dependent on unclear implementation details.
Even if the user-visible search is limited to 'landmarks', does the process pre-create (even if only on-device) fingerprints of many other things as well? If so, it suddenly becomes possible for briefly-active non-persistent malware to instantly find images of interest without the wider access & additional processing it'd otherwise take.
Yup, this is their way of injecting the "phone home" element via an innocuous rationale, "location matching". The global index will of course also match against other markers they deem worthy of matching, even if they don't return that to the user.
Honestly, why the hell would Apple bother with such a contrived and machiavellian strategy to spy on their users?
They literally own the code to iOS. If they wanted to covertly track their customers, they could just have their devices phone home with whatever data they wanted to collect. Realistically there would be no way to know if this was actually happening, because modern devices emit so much encrypted data anyway, it wouldn’t be hard to hide some nefarious in all the noise.
Time Cook isn’t some Bond villain, sitting in a giant chair, stroking a white cat, plotting to take over the world by lulling everyone into a false sense of privacy (I mean Zuckerburg already did that). Apple is just a large giant corporation that wants to make money, and is pretty damn open about that fact. They clearly think that they can make more money by doubling down on more privacy, but that doesn’t work if you don’t actually provide the privacy, because ultimately, people are really crap at keeping secrets, especially when a media group would happily pay for a story, even at Apple.
Yes my thoughts as well. The tech was so expensive I guess that they had a need to test / run it to proof it’s private? I mean the model to find landmarks in your photos could run locally as well or? Ok I’m not 100% sure here.
If this was a feature on its own then it would not be popular.
Citing national security, some danger will justify its existence.
Apple alone does not control and dictate what goes in, once you reach their level of size and wealth that exceed even developed countries, you ultimately cannot be the controller of your destiny purely as a profit orientated corporation.
Very likely yes. Why else would they add a feature that incurs costs for them as an update, at no cost to the users (and not even make a fuss about it)?
It is obvious they are monetizing this feature somehow. Could be as innocuous as them training their AI dataset, or feeding into their growing ad business (locations and other things identified in the photos), or collaboration with law enforcement for various purposes (such as notifying the CCP about people's Winnie-the-Pooh memes), or a lot more ominous things.
> Very likely yes. Why else would they add a feature that incurs costs for them as an update, at no cost to the users (and not even make a fuss about it)?
Erm, you’re aware of the whole Apple intelligence thing right? An entire product that costs Apple money, provided at “no cost” to the user (if you had an iPhone 15). Also every feature in an OS update has a costs associated with it, and iOS updates have cost money for the best part of a decade now.
Has it occurred to you that reason Apple includes new features in their updates is to provide customers with more reasons to buy more iPhones? Just because feature are provided at “no cost” at point of consumption, doesn’t mean Apple won’t make money in the long run, and selling user data isn’t the only way to monetise these features. Companies have been giving out “freebies” for centuries before the internet existed, and the possibility of large scale data collection and trading was even imaginable.
CSAM scanning takes place on the cloud with all the major players. It only has hashes for the worst of the worst stuff out there.
What Apple (and others do) is allow the file to be scanned unencrypted on the server.
What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match.
That file in question would be able to be decrypted on the server and checked by a human. For everything else it was encrypted in a way it cannot be looked at.
If you had icloud disabled it could do nothing.
The intent was to protect data, children and reduce the amount of processing done on the server end to analyse everything.
Everyone lost their mind yet it was clearly laid out in the papers Apple released on it.
Apple sells their products in oppressive regimes which force them to implement region specific features. E.g. China has their own iCloud, presumeably so it can be easily snooped on.
If they were to add this anti-CSAM feature, it is not unreasonable to think that Apple would be forced to add non-CSAM stuff to the database in these countries, e.g. anything against a local dictatorship/ etc. Adding the feature would only catch the low hanging CSAM fruit, at the cost of great privacy and probably human life. If it was going to stop CSAM once and for all, it could possibly be justified, but that's not the case.
"It only has hashes for the worst of the worst stuff out there." [citation needed]
I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub. I mean, I guess the person could have been lying, but I doubt they would even have been talking about it if the truth was less innocuous.
I can't believe how uninformed, angry, and still willing to argue about it people were over this. The whole point was a very reasonable compromise between a legal requirement to scan photos and keeping photos end-to-end encrypted for the user. You can say the scanning requirement is wrong, there's plenty of arguments for that. But Apple went so above and beyond to try to keep photo content private and provide E2E encryption while still trying to follow the spirit of the law. No other big tech company even bothers, and somehow Apple is the outrage target.
> What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match.
This is not the revelation you think it is. Critics understood this perfectly.
People simply did not want their devices scanning their content against some opaque uninspectable government-controlled list that might send you to jail in the case of a match.
More generally, people usually want their devices working for their personal interests only, and not some opaque government purpose.
Yes this is better than upload the entire photo. Just like virus scan can be done entirely on device, can flagging be local?. If homeomorphic encryption allows similarity matching, does not seem entirely private. Can people be matched?
> I don't understand most of the technical details of Apple's blog post.
I did understand the cited bits, and sorry to say but this could have been an optimistic post ("look at this cool new thing!")
I dislike Apple's anti-hacker (in the HN sense of the word) practices as much as the next person and don't own any Apple device for that and other reasons, but saying "it doesn't matter how you solved the privacy problem, I feel it's not private" doesn't make it true. Because most other people don't understand the cited words either, if they read that far down anyway, this seems like unfair criticism
Homomorphic encryption is something I heard about through a research paper a few years ago.
Back then I understood that an operation like SUM would be able to compute the sum of a list of numbers where each number was encrypted. The way the encryption worked made it possible to add all the values together without decrypting them, and the result ended up being encrypted too in such a way that the owner could decrypt it and have a number with a certain known accuracy.
If Apple is using homomorphic correctly then there should be no way for them to see the data they get from your phone. The other things they mention in the post as ways to prevent leaking of other information through metadata or a side channel.
The fact that this feature was enabled by default isn’t exactly great. Definitely should have been something that the user should have been asked if they wanted to enable after upgrading.
One specific use Apple is making of homomorphic encryption as of iOS 18 (I think) is for spam callers. You get a phone call, your phone sends Apple the encrypted phone number, they run it against their spam caller database, and you get the encrypted spam/not spam response back. They published a bunch of open source code around this functionality a while back.
I imagine Apple might argue that no private information is sent thanks to the use of homomorphic encryption. But Apple’s explanation rings hollow without the user having the ability to verify that this system is working as described.
How can you trust something you don't understand? That must come from "authority" (some person or org that you trust to know about such matters). That authority isn't Apple for many people. While I have cautious trust in Apple's privacy policies, many people don't, and not without reason. Hence, not understanding Apple's technical explanation of an Apple feature you didn't opt in to sharing personal data, increases the feeling of privacy violation (which in turn leads to more distrust).
> Hence, not understanding Apple's technical explanation of an Apple feature you didn't opt in to sharing personal data
But this is the fundamental issue. The author has no idea if personal data is being shared, they’ve made an assumption based on their lack of understanding. It’s entirely possible that all this service does (and arguably likely), is provide a private way for your phone to query a large database of landmark fingerprints, then locally try and match those fingerprints to your photos.
It doesn’t require send up private data. The phone could perform large geographic queries (the size of countries) for batches of fingerprints to be cached locally for photo matching. The homographic encryption just provides an added layer of privacy, allowing the phone to make those queries in a manner that makes it impossible for Apple to know what regions were queried for.
iOS photos already uses databases to convert a photo location into an address, so you can do basic location based searching. That will involve doing lookups in Apple global address database, do you consider that a violation of people’s privacy?
So you understand your own device’s security?
You have no more reasons to trust the security of the Apple device in your pocket than you do of an Apple device in a datacenter IMHO.
> sorry to say but this could have been an optimistic post
> don't own any Apple device
So you don't have any skin in the game, but you're criticizing someone who does?
My blog post is written from the perspective of an Apple user whose trust has been violated. It's nice that you think—from a safe distance—the technology is neat, and maybe it is, but that's irrelevant to the main issue, which is the lack of user consent.
Hacker News unfortunately does not respond to this logic unless it is a company they are trained to hate. We could run the same story reporting Google and Meta's opt-out abuses, and it would also reach the frontpage with just as many comments. Except those comments would be violent condemnation, not apologetics and hand-wringing over whitepaper quotes.
It's tragic, because computing is in a professedly imperfect place right now. Digital privacy is under fire, many payments see a 30% digital service surcharge that is wholly arbitrary, and revolutionary cross-platform standards are being supplanted with proprietary and non-portable solutions that does not benefit any user.
As an American, I am ashamed that our government's dysfunction extends to consumer protection.
To me it seems like a reasonable feature that was, for the most part, implemented with great consideration for user privacy, though maybe I’m too trusting of the description. I mostly think this article is rage-bait and one should be wary of ‘falling for it’ when it shows up on hacker news in much the same way that one should be wary when rage-bait articles show up in tabloids or on Facebook.
It seems likely to me that concerns like those of the article or some of the comments in this thread are irrelevant to Apple’s bottom line. A concern some customers may actually have is data usage, but I guess it’s likely that the feature is off if in low data mode.
I wonder if this particular sort of issue would be solved by some setting for ‘privacy defaults’ or something where journalists/activists/some corporate IT departments/people who write articles like the OP can choose something to cause OS updates to set settings to values that talk less on the network. Seems hard to make a UI that is understandable. There is already a ‘lockdown mode’ for iOS. I don’t know if it affects this setting.
Literally all Apple needed to do was not have it enabled by default. Sending stuff over the network without asking is why trust in Apple is reduced further and further.
Not enabling something by default is pretty close to not having it at all. Accessibility is a reasonable exception where it makes sense to have the features even though they are off by default.
I mostly think the reaction to this article is overblown because it appeals popular ideas here about big tech. I think one should be wary of Apple’s claims about privacy: the reason is competition with Google and so they want users to be distrustful of the kinds of features that Google are better at implementing (I don’t want to say Apple isn’t trying to do the right thing either – if you look at accessibility, the competition was very bad for a lot of things for a long time and Apple was good despite the lack of commercial pressure). But I think one should also be wary of articles that make you angry and tell you what you suspected all along. (eg see the commenter elsewhere who doesn’t care about the details and is just angry). It’s much easier to spot this kind of rage-bait piece when it is targeting ‘normal people’ rather than the in-group.
Apple already communicates home by default. They never even fixed the macOS app signature check that they said they would, and yet people still choose to use the OS.
(And to be clear I’m not even bothered by the signature check)
At a certain point you have to figure that they realize it doesn’t matter short of some government entity forcing them to stop. At the very least the protections they put in place (homomorphic encryption, etc) are more than I think most other companies would ever bother doing.
It’s a reasonable feature, but should nevertheless require opt-in by the user. The opt-ins could certainly be bundled at install/upgrade time to reduce annoyance.
One thing particularly not clear to me is weather ios scan all data in the phone and send it to be part of public index or not.
I see from how the feature works from the UI it seems it's not. If the feature activated by user action does this still constitute as phoning home?
if anyone else had done this then yes probably it's reasonable feature done reasonably. The problem is Apple has spent tens if not hundreds of millions of dollars advertising that they don't do things like this. That stuff stays on your iPhone unlike that other OS run by yucky advertising company. Apple would never siphon your data, because they care and you aren't the product.
Shit like this, reasonable in isolation or not, undermines that story completely. If they are so willing to just outright lie on a massive billboard, what else will they do when profits demand it?
I think I just noticed a similar thing for search that I'm pretty sure was not there before IOS 18.
Going into Settings -> Search there's an option now for "Help Apple Improve Search", enabled by default.
>Help improve Search by allowing Apple to store the searches you enter into Safari(!!), Siri and Spotlight in a way that is not linked to you. Searches include lookups of general knowledge, and requests to do things like play music and get directions.
If it was there before then it was switched on again.
> allowing Apple to store the searches you enter into Safari […] in a way that is not linked to you
From deanonymization work even a decade ago it was clear that your search history will completely unmask you.
I would need lots of details before I believed that their method of storing the data reliably protected my privacy. (But of course that is not what the quote claims.)
This is worse than the OP. What alternative universe is Apple living in where it thinks that defaulting to collecting people’s search histories without consent is OK?
As trivia, on mac os, the photoanalysisd service will run in the background and look through your photos, even if you never open Apple Photos. It can't be disabled unless you disable SIP (system integrity protection) which requires a complicated dance of reboots and warnings. It will reenable if you turn SIP back on.
It seems Apple are very passionate about analysing your photos for some reason, regardless if you yourself are.
I was. First by the md_worker processes that mysteriously started pinning all of my CPU cores after a git clone. Then by the realization that MacOS had built a full-text index of millions of lines of source code (it only took a few hours of my Mac being too hot to touch).
A lot of Apple's defaults are just plain bizarre. Why the hell is Spotlight seeing source code mimetypes and feeding it to the search index?
CSAM could already be part of some local service theoretically. Privacy ended with a requirement to have an account linked to the device (not just icloud). There is no account needed to use a Linux computer.
No. The analysis in question is fully local, used for indexing photos by categories in the Photos app. It is unrelated to any cloud features and not something shared across users.
They are also not using your personal photos to feed the location database, most likely public sources and/or Apple Maps data. If they are relying on GPS-tagged public photos alone, you could probably mess up a system like this by spoofing GPS location en-masse and posting them online for years, but for what purpose?
All kinds of nonsense runs and phones home throughout the os. The thing that annoyed me the most is trying to create an account will phone home to apple, such as setting up a local smtp/imap server on the local network.
This whole thing is reminding me of the outrage over Apple and Google's privacy preserving 'Exposure Notification System' system from the Covid years. It defies intuition that they can alert you to exposure without also tracking you, but indeed that's what the technology lets you do.
Similarly here, it feels like the author is leaning into a knee jerk reaction about invasion of privacy without really trying to evaluate the effectiveness of the technologies here (client side vectorization, differential privacy, OHTTP relays, and homomorphic encryption).
Though I 100% agree Apple should ask the user for consent first for a feature like this.
Of course it is not the whole technology stack, but it is something at least. If your evaluation leads to potential problems, you can create issues right there on the github project!
If you have the capability to actually skillfully analyze this type of crypto, disassembling the binaries from your device (or at the very least, an ipsw for your device) should be trivial.
After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?
"I don't understand most of the technical details of Apple's blog post"
I do:
- Client side vectorization: the photo is processed locally, preparing a non-reversible vector representation before sending (think semantic hash).
- Differential privacy: a decent amount of noise is added the the vector before sending it. Enough to make it impossible to reverse lookup the vector. The noise level here is ε = 0.8, which is quite good privacy.
- OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
- Homomorphic encryption: The lookup work is performed on server with encrypted data. Apple can't decrypt the vector contents, or response contents. Only the client can decrypt the result of the lookup.
This is what a good privacy story looks like. Multiple levels of privacy security, when any one of the latter 3 should be enough alone to protect privacy.
"It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here.
I'm not sure what the right call is here. But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true. There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. But if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
I appreciate the explanation. However, I think you do not address the main problem, which is that my data is being sent off my device by default and without any (reasonable) notice. Many users may agree to such a feature (as you say, it may be very secure), but to assume that everyone ought to be opted in by default is the issue.
I'm not sure I agree -- asking users about every single minor feature is (a) incredibly annoying, and (b) quickly causes request-blindness in even reasonably security-conscious users. So restraining the nagging for only risky or particularly invasive things makes sense to me.
Maybe they should lump its default state into something that already exists? E.g. assume that if you already have location access enabled for Photos (it does ask!), you've already indicated that you're okay with something about this identifying being sent to Apple whenever you take a picture.
My understanding is that Location Services will, among other things, send a hash of local WiFi network SSIDs and signal strengths to a database Apple maintains, and use that to triangulate a possible position for you. This seems loosely analogous to what's going on here with the compute-a-vector thing.
I think it does address the main problem. What he is saying is that multiple layers of security is used to ensure (mathematically and theoretically proved) that there is no risk in sending the data, because it is encrypted and sent is such a way that apple or any third party will never be able to read/access it (again, based on theoretically provable math) . If there is no risk there is no harm, and then there is a different need for ‘by default’, opt in/out, notifications etc.
The problem with this feature is that we cannot verify that Apple’s implementation of the math is correct and without security flaws. Everyone knows there is security flaws in all software, and this implementation is not open (I.e. we cannot review the code, and even if we could review code we cannot verify that the provided code was the code used in the iOS build). So, we have to trust Apple did not make any mistakes in their implementation.
I’m a cryptographer and I just learned about this feature today while I’m on a holiday vacation with my family. I would have loved the chance to read about the architecture, think hard about how much leakage there is in this scheme, but I only learned about it in time to see that it had already been activated on my device. Coincidentally on a vacation where I’ve just taken about 400 photos of recognizable locations.
This is not how you launch a privacy-preserving product if your intentions are good, this is how you slip something under the radar while everyone is distracted.
I think I'm saying: you're not sending "your data" off device. You are sending a homomorphically encrypted locally differentially private vector (through an anonymous proxy). No consumer can really understand what that means, what the risks are, and how it would compare to the risk of sending someone like Facebook/Google raw data.
I'm asking: what does an opt in for that really look like? You're not going to be able to give the user enough info to make an educated decision. There's ton of risk of "privacy washing" ("we use DP" but at very poor epsilon, or "we use E2E encryption" with side channel data gathering).
There's no easy answer. "ask the user", when the question requires a phd level understanding of stats to evaluate the risk isn't a great answer. But I don't have another one.
Notice is always good and Apple should implement notice.
However, "my data is being sent off my device" is incorrect, as GP explained. Metadata, derived from your data, with noise added to make it irreversible, is being sent off your device. It's the equivalent of sending an MD5 of your password somewhere; you may still object, but it is not factually correct to say your password was transmitted.
"Your data" is not actually being sent off your device, actually, it is being scrambled into completely unusable form for anyone except you.
This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device.
I guess it depends on what you're calling "your data" -- without being able to reconstruct an image from a noised vector, can we say that that vector in any way represents "your data"? The way the process works, Apple makes their own data that leaves your device, but the photo never does.
How would you explain client side vectorization, differential privacy and homomorphic encryption to a layman in a single privacy popup so that they can make an informed choice?
Or is it better to just trust that mathematics works and thus encryption is a viable way to preserve privacy and skip the dialog?
Do you consider your data to include non-reversible hashes of your data injected with random noise? I'm not sure I consider that my data. Its also not even really meta-data about my data.
When your phone sends out a ping to search for cellular towers, real estate brokers collect all that information to track everywhere you go and which stores you visit.
Owning a phone is a privacy failure by default in the United States.
You're presenting a false dichotomy between "perfect user understanding" and "no user choice." The issue isn't whether users can fully comprehend homomorphic encryption or differential privacy – it's about basic consent and transparency.
Consider these points:
1. Users don't need a PhD to understand "This feature will send data about your photos to Apple's servers to enable better search."
2. The complexity of the privacy protections doesn't justify removing user choice. By that logic, we should never ask users about any technical feature.
3. Many privacy-conscious users follow a simple principle: they want control over what leaves their device, regardless of how it's protected.
The "it's too complex to explain" argument could justify any privacy-invasive default. Would you apply the same logic to, say, enabling location services by default because explaining GPS technology is too complex?
The real solution is simple: explain the feature in plain language, highlight the benefits, outline the privacy protections, and let users make their own choice. Apple already does this for many other features. "Default off with opt-in" is a core principle of privacy-respecting design, regardless of how robust the underlying protections are.
I don't believe I said or implied that anywhere: 'You're presenting a false dichotomy between "perfect user understanding" and "no user choice."'? Happy to be corrected if wrong.
Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.".
Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.
Sure, but if we follow that line of thinking to its logical conclusion, we must move to a cabin in the woods, 100 miles from the nearest civilization, growing our own food and never connecting our computing devices to anything resembling a network.
> The author themselves looks to be an Apple security researcher
They’re not. Jeff Johnson develops apps (specifically Safari extensions) for Apple platforms and frequently blogs about their annoyances with Apple, but they’re not a security researcher.
Thank you for this comment. I found the author's ignorance to be fairly discrediting, and was surprised to find so many follow up comments equally railing on Apple.
Between the quote you pointed out and:
"One thing I do know, however, is that Apple computers are constantly full of privacy and security vulnerabilities, as proved by Apple's own security release notes"
which just reeks of survivorship bias.
I think the final call of what is right here _shouldn't_ be informed by the linked article.
IMO, enabled by default without opt-in is absolutely the right call when judging between 1: Feature value 2: Security risk 3: Consent Fatigue.
If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
If you disagree with my prior statement because of how Apple locks you into Photos, :shake_hands:.
If Enhanced Visual Search is still enabled by default in lockdown mode, then I think that's worth a conversation.
> I found the author's ignorance to be fairly discrediting
Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption?
> which just reeks of survivorship bias.
What does that even mean in this context?
> 1: Feature value
What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.
> If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120
Lockdown mode is basically for famous people and nobody else.
Enhanced Visual Search was enabled despite my default lockdown mode. I worry about enhanced visual search capabilities much less than several of the other risky features that lockdown mode disables, but was a bit surprised by the default opt-in in my lockdown mode phone.
This sounds exactly like that CSAM "feature" they wanted to add but created a huge outrage because of how incredibly invasive it was.
It sounds like it only needs a few extra lines of code to get exactly what they wanted before, they just packaged it differently and we all fell for it like frogs getting boiled in water.
The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!"
The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device.
BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.
I’m deeply familiar with all of these techniques, the core issue here is informed consent which they have not obtained.
Furthermore, Apples privacy stance is generally a sham as their definition of “human rights” doesn’t extend to China. Which either means Apple doesn’t respect human rights, or they don’t view Chinese people as human.
Apple follows the law. First you need to get the Chinese government to respect those rights. The only other choice is to stop doing business entirely in the country.
That's not really fair; Apple's in a sticky wicket when it comes to the Chinese government, and they're not the only ones.
The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.
The nearest neighbour search is sharded, which apple's blog admits is a privacy issue, which is why they're running the DP and OHTTP parts.
If apple were to add additional clusters that match "sensitive" content and endeavour to put them in their own shards distinct from landmarks, they defeat the homomorphic encryption part while still technically doing it.
The DP part can be defeated with just statistics over time; someone with any volume of sensitive content will hit these sensitive clusters with a higher likelihood than someone generateing noise injected fake searches.
The OHTTP part can be defeated in several ways, the simplest of which is just having a clause in a non-public contract allowing apple to request logs for some purpose. They're paying them and they can make up the rules as they go.
This must be the first consumer or commercial product implementing homomorphic encryption is it not?
I would be surprised if doing noisy vector comparisons is actually the most effective way to tell if someone is in front of the Eiffel tower. A small large language model could caption it just as well on device, my spider sense tells me someone saw an opportunity to apply bleeding edge, very cool tech so that they can gain experience and do it bigger and better in the future, but they're fumbling their reputation by doing this kind of user data scanning.
> This must be the first consumer or commercial product implementing homomorphic encryption is it not?
Not really, it's been around for a bit now. From 2021:
> The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1).
I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.
I have an idea: send an encrypted, relayed, non-reversible, noised vector representation of your daily phone habits and interactions. That way you can be bucketed, completely anonymously of course, with other user cohorts for tracking, advertising, and other yet-to-be discovered purposes.
It's a great privacy story! Why would you have a problem with that?
What would be the value to the user in your scenario? In the photos app real scenario, it’s to enable a search feature that requires pairing photos with data not on the phone. (I understand you’re being sarcastic.)
They don't "have your data," even at an aggregated and noised level, due to the homomorphic encryption part.
Restating the layers above, in reverse:
- They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means)
- Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part)
- Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.
I am bit bit confused: Data is being sent to Apple, in such a way that it can not be traced back to the user. Apple does some processing on it. Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....
There is a way to perform processing on encrypted data so the result is also encrypted and the person doing the processing never knows anything about the data that was processed on or the result (which can only be decrypted by the user with the original encryption keys)
> Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted
And the way the data comes back to you is via the third-party relay which knows your IP but nothing else
> - OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
I don't have a list on hand, but at least Cloudflare and Akamai are part of the network hops. Technically you only need 2 hops to make sure no origin or data extraction can be done.
>There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk.
It is simple for any user to evaluate risk the risk of their data being breached on 3rd party servers when their data isn't being sent off the device - there is none. It is only when corporations insist that they are going to send the data off your device whether you like it or not that evaluating risk becomes necessary.
Regarding HE: since the lookup is generated by the requestor, it can be used as an adversarial vector, which can result in exfiltration by nearest neighbor (closest point to vector) methods. In other words, you can change what you are searching for, and much like differential power analysis attacks on crypto, extract information.
The best you can hope is integrity and security until your information reaches the destination but to assume that Apple or the U.S government cannot decipher the information you sent it or use it against you(i.e. set a person of interest as "landmark" and find out who's iPhone matches that "landmark) you must be foolish.
It's no longer a conspiracy. I think we are all over past that time(i.e with Snowden and Wikileaks). We live in a surveillance world and "They're guarding all the doors and holding all the keys".
The right call is to never send any data from the device to anyone unless the user explicitly tells the device to do it.
The only thing the device should do is whatever its user tells it to do.
The user didn't tell it to do this. Apple did.
> But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true
Irrelevant. It was never about privacy to begin with. It was always about power, who owns the keys to the machine, who commands it.
Vectorization, differential privacy, relays, homomorphic encryption, none of it matters. What matters is the device is going behind the user's back, doing somebody else's bidding, protecting somebody else's interests. That they were careful about it offers little comfort to users who are now aware of the fact "their" devices are doing things they weren't supposed to be doing.
Complete nonsense. *All networked devices do things behind their users back* at this point, and have for years, and do not ask for consent for most of it. And users would REJECT granular opt-in as a terrible UX.
Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.
> if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
Well I want them to fuck off.
Hidden in your commentary here is the fact that the vector representation of the image is _the contents of the image_. It very well may be that they cannot reverse the exact image. But it’s still a representation of the image that has to be good for something. Without being too familiar I would be willing to hazard a guess that this could include textual labels and classifications of what is in the image.
I don’t give a shit how good your internal controls are. I don’t have anything particularly interesting to hide. I still do not want you taking my pictures.
If you read the research you'd know that they don't have access to the vector either. They never decrypt the data. All operations on their server are done directly on the encrypted data. They get 0 information about your photos. They cannot even see which landmark your vector was closest to.
> "It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here
I don’t think that that’s what the author is saying at all, I think he’s saying that Apple should let the user decide for themself if they want to send all this shit to Apple, freedom for the individual. They’re not saying “I dunno”
how about they don't send anything about my photos to their servers and i get to keep my shit on my own device
i suppose we're past that to the point where techbros like you will defend personal data exfiltration because.. uhh idk? trillion dollar corporation knows best?
So what? Why should the application talk over the Internet to begin with? And why isn't that functionality off by default under a settings option that clearly warns the user of the consequences? I think you're missing the forest for the trees here.
And the claims that this is good privacy/security are not at all obvious either. And who are those third-parties anyway? Did you verify each one of them?
Quantum makes the homomorphic stuff ineffective in the mid-term. All they have to do is hold on to the data and they can get the results of the lookup table computation, in maybe 10-25 years. Shouldn't be on by default.
What makes you think that this is the biggest problem if things like AES and RSA are suddenly breakable?
If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that.
In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.
The right call is to provide the feature and let users opt-in. Apple knows this is bad, they've directly witnessed the backlash to OCSP, lawful intercept and client-side-scanning. There is no world in which they did not realize the problem and decided to enable it by default anyways knowing full-well that users aren't comfortable with this.
People won't trust homomorphic encryption, entropy seeding or relaying when none of it is transparent and all of it is enabled in an OTA update.
> This is what a good privacy story looks like.
This is what a coverup looks like. Good privacy stories never force third-party services on a user, period. When you see that many puppets on stage in one security theater, it's only natural for things to feel off.
That’s starting to veer into unreasonable levels of conspiracy theory. There’s nothing to “cover up”, the feature has an off switch right in the Settings and a public document explaining how it works. It should not be on by default but that’s not a reason to immediately assume bad faith. Even the author of the article is concerned more about bugs than intentions.
It's not that binary. Nobody is forcing anything, you can not buy a phone, you can not use the internet. Heck, you can even not install any updates!
What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree.
When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it.
It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.
It is new. It is fancy. It is not clear where HE/DP is being used, it depends if the code is written using the Swift toolkit, but even that has paths for exfiltration if used incorrectly. They claim they are using DP in Photos as stated in the article here:
But the fact remains they are looking at your pictures. I do not trust them for one fleeting fart in the wind on this. Think about it for a hot second: HE/DP allows you to perform operations on the data without knowing the data, but what if someone goofs an operation and it ends up returning the actual data?
Sorry, not buying it. Crypto is hard to get right, and when it is monetized like this for "new features", it is wildly unnecessary and exposes users to more risk.
> what if someone goofs an operation and it ends up returning the actual data
That's kind of like saying "they can take a picture of your face and transmit it, but what if someone goofs an operation and sends your actual head instead".
Any encrypted data the server has cannot 'accidentally' be decrypted, and as someone else explained in this thread they only send encrypted vectors describing features of the image (building, tree, etc) and not the actual image. It's certainly not a fact that "they are looking at your pictures" [1].
[1] "they" being Apple; the other guys could have backdoored the whole OS and Apple employee's computers for all I know
So basically - You take a picture. Apple encrypts it and uploads it to their server. The server matches the (still encrypted) picture to a database and tells your device "this picture contains the Eiffel Tower". Later when you search for Eiffel Tower on your device the photo pops up.
Is the complexity and security risk really worth it for such a niche feature?
It's also funny that Apple is simultaneously saying "don't worry the photo is encrypted so we can't read it" and "we are extracting data from the encrypted photo to enhance your experience".
They don’t send the photo. They send some encrypted metadata to which some noise is added. The metadata can be loosely understood as “I have this photo that looks sort of like this”. Then the server takes that encrypted data from the anonymized device and responds something like “that looks like the Eiffel Tower” and sends it back to the device. The actual photo never goes to the server.
With the added caveat that HE is magic sauce - so the server cannot see the metadata (cropped/normalized image data), and doesn't know how much it does or does not look like the Eiffel Tower.
Because it turns out that mathematicians and computer scientists have devised schemes that allow for certain computational operations to be performed on encrypted data without revealing the data itself. You can do a+b=c and it doesn’t reveal anything about what a and b are is the intuition here. This has been mostly confined to the realm of theory and mathematics until very recently but Apple has operationalized it for the first time.
Not really. It's more like apple runs a local algorithm that takes your picture of the Eiffel tower, and outputs some text "Eiffel tower, person smiling", and then encrypts that text and sends it securely to apples servers to help you when you perform a search.
Locally, a small ML model identifies potential POIs in an image.
Another model turns these regions into a series of numbers (a vector) that represent the image. For instance, one number might correlate with how "skyscraper-like" the image is. (We don't actually know the definition of each dimension of the vector, but we can turn an image that we know is the eiffel tower into a vector, and measure how closely our reference image and our sample image are located)
The thing is, we aren't storing this database with the vectors of all known locations on our phone. We could send the vector we made on device off to Apple's servers. The vector is lossy, after all, so apple wouldn't have the image. If we did this, however, apple would know that we have an image of the eiffel tower.
So, this is the magic part. The device encrypts the vector using a private key known only to it, then sends this unreadable vector off to the server. Somehow, using Homomorphic Encryption and other processes I do not understand, mathematical operations like cosine similarity can be applied to this encrypted vector without reading the actual contents of the vector. Each one of these operations changes the value, which is still encrypted, but we do not know how the value changed.
I don't know if this is exactly what Apple does, I think they have more efficient ways, but theoretically what you could do is apply each row in your database to this encrypted value, in such a way that the encrypted value becomes the name of the POI of the best match, or otherwise junk is appended (completely changing the encrypted value) Again, the server has not read the encrypted value, it does not know which row won out. Only the client will know when it decrypts the new value.
This discussion got quite long without anyone mentioning the novel technical implementation paper "Scalable Private Search with Wally". Kudos to Apple for researching this, but shame on them for enabling this by default.
As a somewhat homomorphic encryption scheme, each query releases some bits of information on what you searching for to avoid using the whole database. Subsequent queries from a given user will generally be correlated enough to build a tracer. Governments or other powerful enough actors can pierce the proxy veil, heck the tracer will be able to deanonymize you with enough queries recorded.
How many queries? For me it is too tedious to work out the math from the differential privacy definitions and I already know the landmarks around me: I don't want such a feature.
Very curious here as I haven’t seen any papers demonstrating attacks against the differential privacy systems proposed by Apple or Google that successfully deanonymize data. Such an attack in even a test database would be very interesting.
Do you have any papers you can cite about this entropic leakage you’re describing?
The very difference between somewhat and full homomorphic encryption hinges on this leakage as explained in the paper. The definition of differential privacy as well. They directly admit to leaking a certain amount of information by stating that they apply differential privacy with those given parameters. The issue I am talking about is that such concerns are applied on a single query but correlations across query (the things that actually happen with metadata) aren't considered in the delta-epsilon differential privacy model, by definition.
I don’t even use iCloud Photos and this was on by default. Very bad move by Apple to ship my photos off my device, without my permission, in any shape or form, I don’t care.
It’s funny how the wordsmiths come out to defend Apple here.
Your “photos” aren’t shipped off-device without your knowledge, just an arbitrary blob of ”metadata” that you can’t audit describing everything about that photo. :)
It’s sort of like “I don’t want my WiFi router uploading my house online!” And someone replying “it’s not your physical house, just a map of the house and the real time location of everyone in it! The house never moves!”
Another setting that surprised me with being turned on by default apparently on macOS 15 is System Settings - Spotlight - "Help Apple Improve Search":
"Help improve Search by allowing Apple to store your Safari, Siri, Spotlight, Lookup, and #images search queries. The information collected is stored in a way that does not identify you and is used to improve search results."
No, this is not on by default. After system install at first boot it asks if you want to help improve search and it describes how your data will be handled, anonymized, etc. If you clicked on yes it is on. There is a choice to opt out.
I was on by default for me, both after the macOS 14 -> 15 upgrade and after installing macOS 15 cleanly. I wonder if they ask for consent in some regions only.
At this point, Mac Mini M4's are cheap enough and capable enough to just purchase two: one for off-line use, another on-.
Perhaps this is marketing genius (from an AAPL-shareholder POV)?
----
I'm laughing at the insanity of all this interconnectivity, but an NDA prevents me from typing the greatest source of my ironic chuckles. Described in an obtuse way: a privacy-focused hardware product ships with an undisclosed phone-home feature, letting the feds see every time you use the product (to produce a controversial product, at home).
Kick in my fucking door / sue me: it'll just re-enforce that I'm correct about concessionary-allowances...
Computer are inexpensive enough to own both on- & off-line hardware.
----
Even privacy-focused hardware manufacturers will allow undisclosed usage tracking (in order to continue existing, themselves). In my example, the OEM delivers a physical product which allows users to make tangible objects, at home. Every time you launch the hardware's control software (to make more controversial objects), it phones home.
Now they probably don't care about you personally, but you'd be surprised how many people are connected indirectly to a person of interest. Google "Maher Arar" for the horrific consequences of false positives.
Wrong. Anything that makes any network request is by definition a privacy leak. The network itself is always listening, and the act of making any connection says that you are using the computer, and where, and when.
> On macOS, I can usually prevent Apple software from phoning home by using Little Snitch. Unfortunately, Apple doesn't allow anything like Little Snitch on iOS.
On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
Apple does publish IP ranges for different services, so it's theoretically possible to block 17.0.0.0/8 and then open up connections just for notifications and security updates, https://support.apple.com/en-us/101555
An iOS "local VPN" could definitely block all traffic to Apple IP ranges. But it lacks the ability to associate traffic with the originating process/framework. Like if, for example, I wanted to only allow iMessage to talk to Apple but nothing else. This is what Little Snitch and other software gives you on macOS/Linux/etc.
But even blanket blocking of all Apple IP ranges probably wouldn't do anything here. As documented, your device sends noise injected image vectors to OHTTP relays and doesn't contact Apple directly. By definition those relays are operated by 3rd parties. So if you consider this type of data "phoning home" you'll need to find the IPs of all of OHTTP relays iOS uses. (or block the traffic that looks up the OHTTP relays).
Apple's "enterprise networking" guide lists 3rd-party CDNs as subdomains of apple.com, which usually resolve to akamai or cloudflare subdomains. This allows those dynamic IPs to be blocked via dnsmasq ipset rules. In theory, they could use similar subdomain resolution for the OHTTP relays.
Since iOS was derived from macOS, perhaps Apple could restore the link between network traffic and process.
It looks sensible assuming that Little Snitch has some high level manager agent inside Apple manipulating the company making these kind of sneaky attacks on customers' privacy that drives the sales of Little Snitch. On the end they will also make them to buy Liitle Snitch for lots of millions or billions for elimination so they can attack customers freely afterwards. Little Snitch hidden agents are smart!
I do not assume that Apple managers are that degenerate idiots pushing through trust eroding marginal idiocy like this.
>On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
NetGuard firewall doesn't run on iOS, so there's no point in comparing to Apple. For those on Android, NetGuard is open-source, https://github.com/M66B/NetGuard
If they advertise "What happens on your iPhone, stays on your iPhone.", these kinds of options must be off by default. Since they are on, I can consider this one and their other statements about privacy lie.
I'm moving my family out of apple photos, self hosted options have come a long way. I landed on immich [0] and a caddy plugin that allows for PKI certificate for account access while still allowing public shared URLs [1]*
There's also LibrePhotos which is packed with features but doesn't have as much polish as immich. They do however have a list of python libraries that can be used for offline/local inference for things like having an image model create a description of a photo that can benefit the full text search. [2]
i like to self host but i’d never do it for something critical like photos. what if you’re hit by a bus next month? Then your family (probably not tech savvy) need to figure out how to migrate all their stuff to a commercial offering - while dealing with all the emotional stuff after death. it’s just added stress for your loved ones.
100% private is nice but not practical for something like photos in my opinion. unless you have someone in the family that can take over after you’re gone and is very technical.
Sure, I use Ente and have iCloud Photos turned off, but TFA is referring to what happens in the Apple Photos app, which happens even if you have iCloud Photos turned off.
> I have concerns about how women would be affected by law enforcement who might use the location data to target abortions.
I believe this risk already exists in the form of LE request with court order to see the contents of an iCloud account, such as conversations and photo metadata. Advanced Data Protection exists to eliminate this particular vector - Apple no longer has account recovery information like private keys escrowed, so there's no way they can provide it if requested.
This would not increase such a risk since the network traffic is effectively VPN'd for anonymization, clients send false data periodically to break profiling or behavioral analysis, and the requests and corresponding results are not comprehensible to Apple's server due to the HE.
police using photos/location to figure out that a woman from Texas went to an address associated with an abortion clinic, and wow Texas has bounties, noooo potential for abuse there.
you've really been shilling apple and even obnoxiously giving multiple replies to the same comment, what's your problem?
The referenced Apple blog post[1] is pretty clear on what this feature does, and I wish the author at lapcatsoftware (as well as folks here) would have read it too, instead of taking the blog post as-is.
Apple has implemented homomorphic encryption[2], which they can use to compute distance metrics such as cosine similarity without revealing the original query/embedding to the server. In the case of photos, an on-device model is first used to detect if a landmark may be possibly present in the image, and then sends an encrypted embedding[3] for the image region containing said landmark, which is evaluated on the server using HE techniques and then the encrypted results are sent back.
I'm sure someone will come along and say Apple does none of these things; in which case, said commenter should probably not use Apple devices since, there is no reason to trust the toggles for "Enhanced Visual Search", and perhaps more importantly, the one for "Advanced Data Protection" either. However, I rarely see any other major company researching ML and HE together, so that alone is a sufficient positive indicator in my eyes.
(It's interesting that this is also being downvoted. Please go ahead, I can't stop you :P but please also write the parts that you disagree with. Thank you.)
> please also write the parts that you disagree with. Thank you
The problem invoked by the article is that data is being sent back to Apple by default. Saying "it's fine because it's encrypted" and "don't use apple if you're not fine with that" doesn't help.
The post complains about a product that stored sensitive customer content locally now sends that data to Apple, and given the combination of abuse on privacy and horrendous, generalized security failures that we've seen across the industry, those concerns seem genuine. Your comment is very dismissive of these concerns, which would explain why it's being down voted.
Apple also makes a mandatory API call to captive.apple.com from every device with a web view, just about every time they connect to a network.
If someone is willing to take a hardline stance that a vendor should inform the user and require opt-in consent for every type of exchange, they are going to have to run platforms specifically targeted toward that mindset.
You're extensively describing the technical implementation while missing the fundamental issue: Why is Apple enabling this feature by default for what is, essentially, a luxury photo search feature?
Let's break this down:
1. Nobody is questioning whether Apple's technical implementation is sophisticated or secure. It clearly is.
2. Nobody is suggesting the privacy toggles don't work. They do.
3. The issue is about Apple deciding that automatically sending data about users' photos (regardless of how securely) is an acceptable default for a feature many users may never want or need.
Consider the value proposition here: Apple invested significant engineering resources into complex homomorphic encryption and differential privacy... so users can search for landmarks in their photos? And they deemed this feature so essential that it should be enabled by default?
This feels like using a golden vault with military-grade security to store grocery lists. Yes, the security is impressive, but that's not the point. The point is: Why are my grocery lists being moved to the vault without asking me first?
A privacy-respecting approach would be:
"Hey, would you like to enable enhanced landmark search in your photos? We've built some really impressive privacy protections to make this secure..."
Instead of:
"We've already started analyzing your photos for landmarks because we built really impressive privacy protections..."
The sophistication of the technology doesn't justify making it the default for what is ultimately an optional convenience feature.
Thank you for posting the apple blog post. As usual, it's really interesting research, and it's fascinating to see how they solve potentially gnarly privacy issues.
That's not the point of the outrage though (at least not for me). They enabled by default a feature that analyzes my pictures (which I never upload to iCloud) and sends information about them to their (and others') servers. That is a gross violation of privacy.
To be clear, I don't care about any encryption scheme they may be using, the gist is that they feel entitled to reach into their users' most private data (the photos they explicitly said they don't want to upload to iCloud) and "analyze" them.
This is the same as that time Microsoft enabled OneDrive "by mistake" and started slurping people's private documents and photos saved in default locations (arguably worse since no one takes pictures with their PC's webcams).
If you really didn't want your photos to be analyzed, would you be using an iPhone? Or any modern smartphone? Google photos doesn't have nearly the privacy focus and no HE whatsoever but I rarely see that mentioned here. It almost seems like Apple gets held to a higher standard just because they have privacy preserving initiatives. Do you use a keyboard on your iphone? You may not have heard but apple is tracking which emojis you type most often [0] and they get sent to apple servers.
Just from memory when the scheme came up in earlier discussion.
The system is essentially scanning for the signature for some known set of images of abuse so that it aims to capture abusers who would naively keep just these images on their machines. (It can't determine if a new image is abusive, notably).
It's conceivable some number of (foolish and abusive) people will be caught this way and those favoring a long dragnet for this stuff will be happy. But this opens the possibility that a hacker could upload an image to an innocent person's computer and get that person arrested. Those favoring the long dragnet will naturally say the ends justify the means and you can't make an omelet without cracking a few eggs. Oh, "think of the children".
Edit: Also worth adding that once a company is scanning user content to try to decide if the user is bad, it makes it that much easier to scan all kind of content in all kind of ways for all kinds of reasons. "for the good", naturally.
>The system is essentially scanning for the signature for some known set of images of abuse
Huh? The system is scanning for landmarks, not images of abuse.
>people will be caught this way
Due to the homomorphic encryption, I don't think Apple even knows whether the image matches a landmark in Apple's server database or not. So even if Apple put some images of abuse into its server database (which Apple claims only contains pictures of landmarks), I don't think Apple would know whether there was a match or not.
The Internet Society makes the following recommendations based on the European Commission’s proposal:
That the European Committee introduce safeguards for end-to-end encryption.
That the European Committee prohibits the use of scanning technologies for general monitoring, including client-side scanning
Summary: "Homomorphic encryption (HE) is a cryptographic technique that enables computation on encrypted data without revealing the underlying unencrypted data to the operating process. It provides a means for clients to send encrypted data to a server, which operates on that encrypted data and returns a result that the client can decrypt. During the execution of the request, the server itself never decrypts the original data or even has access to the decryption key. Such an approach presents new opportunities for cloud services to operate while protecting the privacy and security of a user’s data, which is obviously highly attractive for many scenarios."
I don't understand why this feature needs to exist at all. Ostensibly, if someone wanted to use this, they also have location data on, the phone not only knows its location but also which direction it is pointed via the magnetometer.
I understand it is doing feature matching on the image, but if you were taking a picture of the statue of liberty, the phone would already know that from existing signals.
Every time one of these articles pops up about this sort of behaviour but one of the big tech companies, I'm always struck by the differences in the comments you get. I wonder if Google would be getting the benefit of the doubt like apple is currently getting were Google to implement the exact same thing as an opt-out feature?
It doesn’t seem odd to me. I’m more suspicious of Google’s motives and am even more suspicious of companies like TikTok, Amazon, Xiaomi, and Facebook. There’s a privacy spectrum.
I think sometimes the reactions are different because the facts are different. Your counterfactual seems less likely to me than eg Google photos analysing and indexing each image.
It sounds like Apple implemented a feature and went out of the way to preserve privacy. For most users the correct choice here is to enable the feature by default.
Years ago I went to a concert with some friends. We took pictures standing in front of the stage while the staff was setting up. There were some additional pictures of the show. Nothing in any of the pictures had any information regarding the band names. This was taken with an iPhone 6s, so no "intelligence". In the past year or two I had a new "memories" album. The album title was the artist name that we went to see. At first I thought this was cool, then I was concerned, like, wait, how did they get this information?
My guess, the "intelligence" used the GPS info to pull up the venue and with the date figured out who the artist was. The album was the headliner, the group photo was of one of the opening bands.
I'm still on iOS 17, and I don't see any other way to turn this feature off. I have an option to reset suggested memories, but that's it.
It sounds more like a users are given a choice now, which would be a good thing. If it was enabled pre iOS 18 that kind of makes sense that it's enabled by default now.
If your core concern is privacy, surely you'd be fine with "no bytes ever leave my device". But that's a big-hammer way to ensure no one sees your private data. What about external (iCloud/general cloud) storage? That's pretty useful, and if all your data is encrypted in such a way that only you can read it, would you consider that private? If done properly, I would say that meets the goal.
What if, in addition to storage, I'd like to use some form of cloud compute on my data? If my device preprocesses/anonymizes my data, and the server involved uses homomorphic encryption so that it also can't read my data, is that not also good enough? It's frustrating to see how much above and beyond Apple has taken this simple service to actually preserve user privacy.
I get that enabling things by default triggers some old wounds. But I can understand the argument that it's okay to enable off-device use of personal data IF it's completely anonymous and privacy preserving. That actually seems very reasonable. None of the other mega-tech companies come close to this standard.
iCloud is opt in. This should be too. A lot of people are fine with keeping their photos offline-only and syncing with their computers through a cable.
Making it “private” with clever encryption is their job since Apple wants to sell privacy. They aren’t doing it because they are nice or care about us. Plus, code is written by people and people write bugs. How can you tell this is truly bug-free and doesn’t leak anything?
Ultimately, making it opt-in would be painless and could be enabled with a simple banner explaining the feature after the update or on first boot, like all their opt-in features. Making it opt-out is irresponsible to their branding at best and sketchy to their users at worst, no matter how clever they say it is.
No — users should be the ones to decide if “encrypted on remote storage” is a beneficial trade off for them and their particular situation.
I think there’s some weird impulse to control others behind these decisions — and I oppose that relationship paradigm on its own grounds, independent from privacy: a company has no business making those choices for me.
You are free to use such services if you wish; others are free not to use those services.
I have no idea what apps are sharing with what. On Android network access is so ambiguous. There's such fuzzy wording. Like when you are asked for location permission to use bluetooth. Even perms like file system access. I don't know what this extends to. Have no idea what it is doing. I recently set up a new ipad. I failed to even work out where Photos ended up. Couldn't work out what was backed up, and what wasn't. How to opt out of everything and opt in piecemeal. Whether the OS/gadget had AI enhancements, what they were or are, whether the apps would have cloud access or not. In fact for an apple device it bugged me with dialogs from the get go. And bluetooth kept turning itself back on. I would say I am technically savvy, but I was pretty clueless. I was quite keen to try out some of the AI photo tools. Like find pictures of such and such, but I didn't get that far as confusion abound.
So what can Apple do with this assuming they can send anything not just "landmarks".
What metadata about the image do they get?
Are images from the web or social media or device screenshots accessible too?
Seems at the very least they are getting a database of things, objects and shapes. Can cohorts of users be identified? Useful for ad targeting?
They should be able to determine how many devices have a photo of a particular person. Or have a photo of a piece of art. Useful for capturing wanted people and stolen goods! They should see how many times users take a photo of another phone. When and how many users take a photo of a protest. Which side has which colour. Do people take selfies of themselves when sick with a novel pandemic.
Do people take more photos of dogs at Christmas than cats.
Anything that can be vectorized can be identified. But what data do they get and what could they do with it?
> To uphold our commitment to privacy while delivering these experiences, we have implemented a combination of technologies to help ensure these server lookups are private, efficient, and scalable.
Efficiency and scalability have nothing to do with "upholding one's commitment to privacy". This shows they're insincere.
But, is privacy achievable today? I doubt it. People desperately want (or think they want) features that are opposite to privacy, and if you don't give it to them, they're unhappy. I think Apple's fault is in promising something they can't deliver. (That also goes for Brave, Kagi, Duck Duck Go, etc.)
Scott McNealy famously said "You have zero privacy. Get over it." This was in January 1999. 25 years ago!
If they said To uphold our commitment to privacy while delivering these experiences, we have implemented a combination of technologies to help ensure these server lookups are private then it would be fine. It would also be a tautology.
When they add that server lookups are also "efficient and scalable", it means that they have had to ponder the privacy aspects with technical concerns regarding efficiency and scalability, and that therefore, privacy is mitigated.
I think a fair reading of this sentence would be: "we provide a version of 'privacy' that we feel is acceptable, within reasonable efficiency and scalability constraints".
They're not going to dedicate a server per customer for example. Would it make sense to do it? No. But it would be honest to say "because of efficiency and scalability limits, the 'privacy' we provide is relative and subject to breaches". (I actually think that's exactly what their wording is trying to say.)
Going off a tangent, I wonder if the market reveals survival bias: companies and products that did respect privacy practice (e.g. by asking explicit permission) were not able to harness enough user data to compete with other bad players, and as a result, any company would eventually end up like Facebook or go out of business.
Sadly privacy is not a marketable feature, or at least it does not have the ROI as Apple originally believed. I feel the only way to level the play field is to reconsider our regulation framework and treat privacy as a fundamental benefit for consumers.
Apple can do this because there are absolutely no significant consequences. In the classical legal world your only recourse to this invasion of your privacy along with falsified advertising is a "class action suit". It is clear from past class actions suits that (remember the batter thing) this does not prevent Apple from doing the same thing again.
The problem is the granularity. How do millions of people recover damages when a corporation knowingly (knows or should know) it is acting to enrich itself in a significant manner in small amounts. Let us suppose that an abstraction of the damage from this offense can be quantified at $1 per customer. A significant question is whether this action has any possible benefit to Apple. If it does not then once notified of this action, Apple would immediately (as in the next update of IOS or Sequoia) remedy this feature.
So step #1 is someone to send an official letter, perhaps with just a link to this article. Or perhaps someone from Apple is reading these comments and can inform us whether they are aware.
Next state is that Apple is aware of this problem (knows or should know) and ignores it. So are you helpless? Or could you file a claim in small claims court seeking to recover $1 for each photo that Apple has unloaded, requesting that Apple delete all photos as well as all data derived from the photos, plus filing fee.
Next state. You comment on this post as to how to improve the process and make it faster and easier. [And if you oppose this idea, could you please explain your position in a way that helps others understand?]
Just ged rid if icloud and apple photos. slow as f*ck, crash all the time once the library get big enough on my mac. damn slow to scoll in the photos app with lot of picture.
Is it possible to search immich images via spotlight like one can from iCloud photos? I’d love to jump ship away from iCloud but spotlight search is legitimately helpful and I use it many times a week. It’s saved my bacon quite a few times too (didn’t realize I had eg a document as a photo or pdf instead of email).
It sounds like Apple Photos "phones home" by fetching some indices? It doesn't sound like photo data leave the device, but I think Apple's issue is they should have been more clear about how this feature worked and what data is and isn't transmitted.
If it works how I think it works, I actually don't see an issue here. I don't think anyone here expects there phone to not "phone home" to check for OS updates by default?
It's been doing this since long before iOS 18. I first observed this behavior already on my 1st gen. iPhone SE with iOS 11 or maybe iOS 12. Every now and then, especially after having taken new pictures with the camera, I'd see the network activity spinner get going whenever starting Photos - and I've never enabled iCloud backups, but rather explicitly always disabled it everywhere I find a setting to do so. With Wi-Fi it'd be a short one second affair, and on cellular it'd take a few seconds more. I also observed the duration of the activity to vaguely extend with the number of new pictures taken between starting Photos. Curiously, when disabling cellular data permission for Photos and having no Wi-Fi connected, the network activity spinner would still pop up and just get stuck indefinitely without ever finishing whatever it's reporting.
My gut feeling has always been that it's returning digested machine learning feedback for Photos' facial/object recognition.
(add.: since moving from the iPhone 8 and iOS 16 to a modern iPhone and iOS 17/18 this behavior has gone away - at least the network activity spinner doesn't gossip anymore...)
> Allow this device to privately match places in your
photos with a global index maintained by Apple so
you can search by almost any landmark or point of
interest.
Couldn't they just make a k/v database that has coordinates of landmarks and then just pass the lat/long from the picture's EXIF in an API call for lookup in said DB when the user wants to make use of this feature? Isn't this part of why we have geotagging?
I've noticed that too. I think it depends on the topic. Many are biased when it comes to Apple (me too). Some even defend their misbehaving. My emotional reaction to Apple misbehaving is usually anger because I somehow feel disappointed, even betrayed by Apple. "Apple, how can you build such gorgeous hardware and still act so unethical?". This is of course irrational: Apple is a company that does everything to grow.
My view is that a large group of people interested in building companies, tools, etc moved on and only come back when there’s a PR issue that pushes then to comment. What’s left behind is basically the same old /. esque crowd from the internet of old.
I think the user should be prompted to enable new functionality that sends data to the cloud. I think there should be a link to details about which data is being sent, how it is being transmitted, if it is stored, if it is provided to 3rd-parties, and what is being done with it.
Maybe this is exactly what the GDPR does. But it is what I would appreciate as a user.
I have seen how sending metrics and crash dumps from a mobile device can radically improve the user experience.
I have also seen enough of the Google bid traffic to know how much they know about us.
I want to enable sending metrics, but most of the time it's a radio button labeled "metrics" and I don't know what's going on.
Apple also does this thing where, after a major version update (or, seemingly, random incremental updates) it'll just go ahead and reset your app settings for you. So, make sure to check every single privacy option once a month or so, or just give in and let Apple do whatever it wants.
> It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations.
It's a recurring theme in the modern IT industry that the user can somehow never trusted to take their own responsibility for anything. Like, can I please not have the "added security" on my Android phone where it would sometimes ask me for the pattern instead of the fingerprint? I know my own risks, thank you very much, Google. Same for macOS occasionally making me enter the password instead of letting me unlock with touch ID. Same for a bunch of other random software treating me as a complete technologically illiterate idiot with no way to opt out of this hell.
Remember how our devices used to serve us, not the other way around?
I'm genuinely surprised that there are many software developers that defend Apple products as great tool for privacy even though we are not able to inspect their code that is closed.
It's just hypocrisy. I don't know how people can defend Apple as a privacy company when they phone home more than their Google counterparts.
People will say that at least Apple don't have a vested interest in their data since they are not an advertising company like Google or Meta. But Apple collects an absurd amount of data, it is phoning home every second. Some Apple traffic is hard-coded to go through VPNs.
Apple has already experimented with ads in Maps. Shareholders demand infinite growth, so it is only a matter of time before Apple dives into ads themselves.
The only valid point is that Apple does protect your privacy – only towards other companies.
I switched to Apple around 3 years ago because I don't like having my data coupled to Google. But I am regretting my decision, I will probably switch back to Android + Linux instead of IOS + Mac at this rate.
Wow, this was even enabled on our otherwise fairly locked down corporate phones... Normally these features are both disabled and greyed out via "group policies" or whatever Apple calls them.
Settings -> Siri -> Apps (Siri app access) -> [an app] -> Learn from this app
This is on by default for all apps and it “Allows Siri to learn from how you use [an app] to make suggestions across apps”. I found no global setting for all apps.
We need to go back to local-first software. I'd be happy to pay premium for a device, which is truly my, without big tech sniffing all my data or deciding what can I do with it
The big question for me is what data contributes to the global index. I would definitely be unhappy if my own photos (pixels + GPS location) were being added to this global index. For example, if someone takes a picture of my house, could they find the location (reverse image to location lookup) using this service? Does anyone know what comprises this global index?
Don't be too worried about what they collect per se they all do and you mostly won't know. Let's worry about how they manage and use it. When it comes to Apple specifically - homomorphic encryption is "common" and well done - it how do they use this encrypted data is the key...
Apple decided to siphon user photo derived data without prior consent. The purpose, capacity and risk of said data being in others’ hands we are expected to take their word for. All while not informing users in advance or trusting users to themselves decide to start the siphoning. By doing this they have violated some users trust.
I kept telling people here and on Reddit that Apple is using the whole idea of "Privacy First" on Apple devices as a marketing gimmick. The company has began introducing bloatware into their product and rivaling Windows bloatware.
Yet, people perception on Apple bloatware here seem to be, "that's fine".
> From my own perspective, computing privacy is simple: if something happens entirely on my computer, then it's private, whereas if my computer sends data to the manufacturer of the computer, then it's not private, or at least not entirely private. Thus, the only way to guarantee computing privacy is to not send data off the device.
You can't even turn on and use a new apple product unless you phone home and ask apple for permission. I'm not sure why this is newsworthy. Apple products are known to phone home more than just about anything else. It would be more surprising if apple photos didn't phone home.
I don't mind the whole use of ML based features even if it requires elaborate schemes to run some computation in the cloud in some privacy preserving manner.
However I do see that Apple marketing campaign tried to fool people into thinking somehow non Apple are not private and only Apple provides privacy.
Worth noting that you need iCloud active in the first place to opt-out of this feature, so it is still opt-in, since it is only ever defaulted to being on for users who proactively signed into iCloud.
Ah yes, you're right. I spoke to soon. I just checked on the latest Beta simulator and, sure enough, it is checked, with a fresh install and no iCloud account.
At least that proves the service is accessible anonymously (i.e., without an identifying iCloud token)
So it sends your photos to be indexed on Apple servers. Turned on by default.
This is probably done to compete with Google Photos which has a great photo search by word feature.
With that said, Apple can use whatever privacy measures to protect user data. But at the end of the day, a subpoena can easily force them to hand over data.
The best privacy measure is to just not have the data. I guess indexing photos offline in phone is not very feasible yet.
> This is probably done to compete with Google Photos which has a great photo search by word feature.
Apple Photos is already capable of searching your photos by word/content fully offline on-device.
Google Photos is unable to search your photos by word/content fully offline on-device. Hell, it can't even search by file name if you don't sync your photos to the cloud!
I don't think Apple has to worry about Google at all.
> I guess indexing photos offline in phone is not very feasible yet.
There has been a form of on device indexing since at least iOS 12. My understanding is it performs a basic indexing typically overnight when the phone is charging and allows one to perform searches like "dog" or "car" and pull up matching images.
it literally says it uses global search index on the label below the check mark. It seems more than likely that (now or at least in the long run) they will use user data to enhance this index.
It's called "Enhanced Visual Search" and they described it like this:
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
I guess it's useful if you get very lost and don't recognize the pyramids right in front of you, and forgot you have a GPS in the phone.
I'm disappointed in the discourse around Homomorphic Encryption and Differential Privacy here. As someone who briefly studied adjacent subjects these tools excite me more than they scare me.
We trust TLS with our SSNs and CC numbers, hopefully one day we can trust secure multiparty computation too.
Is this really an apt comparison? I understood the trust in TLS to be built on open RFCs and implementation stacks. Even then, whenever I send private data, I take specific steps to verify I am using that trusted stack. That is not the experience described in the article.
> I take specific steps to verify I am using that trusted stack
I would be very interested to hear what these specific steps are. How do you make sure that this TLS stack really does implement the RFC? How do you know that each connection is indeed encrypted, and it doesn't start sending plaintext after, say 30 days of usage?
Let’s step back and go through the thought process of the team that’s implementing this feature. If they leave the feature disabled by default most likely casual users will never use it because they won’t be able to find the setting buried under all the menus. Thus after adding all their privacy layers the team felt that it should be safe enough to enable by default while remaining true to the company’s ethos of privacy.
Now what would you have done differently if you were in charge of rolling out such a feature? While I don’t like Apple phoning home without my consent, in this case my photos are not being sent to Apple; only anonymized hashes used to match well known landmarks.
The author of the article goes on to show his bias against Apple with phrases like “You don't even have to hypothesize lies, conspiracies, or malicious intentions on the part of Apple to be suspicious of their privacy claims” or “Apple's QA nowadays is atrocious.”
Or this rich quote “I never wanted my iPhone to phone home to Apple.” What smartphone or computing device never communicates back to their company servers? Even when I use open source libraries I have to communicate with repositories to check for dependencies. Does the author hide every online transaction he makes? Never using email or text messages or cloud services which will leak his private information? Unlikely. He just wants to grind his axe against Apple.
So let’s step back and look at it reasonably and see if Apple is trying to erode our privacy with this feature. I personally don’t see this particular feature as being harmful, but I will thank the overzealous author of bringing it to my attention and I’ll be disabling the feature since I don’t need it. This feature is no where near as invasive as the CSAM detection tool that was proposed, which did warrant critical discussion.
Let’s let the team, undoubtedly one of many with the unenviable task of making Apple Intelligence relevant, who rolled out the feature get their yearly bonus and move on to discuss more enlightening topics such as as the merits of keeping tabs on the urination habits of the crew on the ISS via the Mac menubar.
If Apple can prompt me 4700 times after I’ve said no each time to enable Apple TV and Apple Arcade and Apple Fitness subscriptions, they can prompt users to enable a new feature if they wish.
Now "What happens on your iPhone stays on your iPhone" seems like it deserves the Lindt chocolates defense: "exaggerated advertising, blustering, and boasting upon which no reasonable buyer would rely."
Tbh whilst this should be improved it is more like those who lives and works in free democratic countries with market access by and large complained about their imperfect capitalism, undemocratic and not social enough country. May be it is.
But I do not think so by and large.
Keep on fight and Lee on pushing. At least Apple has a slogan you can push on.
Where are you? I'm in Poland and I just disabled it both on phone and tablet - the setting switcher is "conveniently" placed at the bottom of Photos settings page
It is frankly nothing short of amazing that Apple ships things like homomorphic encryption, and differential privacy, and client-side vectorization, and encrypted vectors, at the scale that they inhabit... and they still get a bad report card back from consumers about privacy.
Comparing Apple and Google, or Apple and Microsoft, it seems to me that Apple's track record on these issues is actually not as bad as public opinion might suggest. Meta doesn't even make the list for comparison, and neither does Amazon.
It makes me wonder if picking privacy as a strategy is workable in the first place. People trust TLS; people use banking apps on their phone now without thinking. I remember in 2008 or so when people still didn't quite trust SSL.
I'm not sure if Apple will be able to bridge the gap here, though, if _all_ of their competition simply chooses not to ship those features. Do customers know the difference? Do they... care? In theory they want their data to be private, yes. But if they are not willing to educate themselves to perform their counterparty obligation in fulfilling "informed consent," there may be no action Apple could take to avoid catching the same bad rap everyone else does.
Consumers seem to think well of their pro-privacy standpoints, even if the devil is in the details with regards to how effective it might be.
The people giving “poor reports” are often the hardcore tech users (or LARPers) who grew up in a different era and mindset and are slowly being shoved out.
How did they do it? The idea for homomorphic encryption has been around a long time, but as far as I am aware the time+space requirements have found to be tremendous, thus rendering any serious implementation of it impractible.
If they managed to do it, they should open-source the code. If they made a research breakthrough, they should publish. Doing either of those things would give me the peace of mind that I need.
> It is frankly nothing short of amazing that Apple ships things like homomorphic encryption, and differential privacy, and client-side vectorization, and encrypted vectors, at the scale that they inhabit... and they still get a bad report card back from consumers about privacy.
Personally, I don't shy away from criticizing Google, but that's not the point. Apple makes big claims about their privacy practices that neither Google nor Microsoft make, it would be bizarre to hold Google or Microsoft to claims and standards that Apple set for themselves.
> Comparing Apple and Google, or Apple and Microsoft, it seems to me that Apple's track record on these issues is actually not as bad as public opinion might suggest. Meta doesn't even make the list for comparison, and neither does Amazon.
> It makes me wonder if picking privacy as a strategy is workable in the first place. People trust TLS; people use banking apps on their phone now without thinking. I remember in 2008 or so when people still didn't quite trust SSL.
> I'm not sure if Apple will be able to bridge the gap here, though, if _all_ of their competition simply chooses not to ship those features. Do customers know the difference? Do they... care? In theory they want their data to be private, yes. But if they are not willing to educate themselves to perform their counterparty obligation in fulfilling "informed consent," there may be no action Apple could take to avoid catching the same bad rap everyone else does.
I've said this elsewhere, but what I really dislike about Apple's strategy regarding privacy is that they treat privacy as a purely technological problem that could be solved if only we had better technology, but they ignore that a huge component of why users have been subjected to so many flagrant privacy violations is because they have zero visibility and zero real control over their computing experiences. Apple would very much like to retain their iron grip on what users are allowed to do on their platforms, as they make a ton of money off of their control of the platforms in various ways, so they have a huge incentive to make sure we're all arguing about whether or not Apple is better or worse than Google or Microsoft. Because sure, I do believe that if we held Google to the same privacy standards that Apple currently has, it would probably kill Google. However, if Apple and Google were both forced to give more transparency and control to the users somehow, they'd both be in a lot of trouble.
Despite all of this effort, I think that user trust in the privacy of cloud computing and pushing data out to the internet will only ever go down, because attacks against user privacy and security will only ever continue to get more and more sophisticated as long as there are resourceful people who have a reason to perform said attacks. And there certainly always will be those resourceful people, including in many cases our own governments, unfortunately.
Holy crap! Enabled by default! Thank you for letting everyone know.
“Enhanced Visual Search in Photos allows you to search for photos using landmarks or points of interest. Your device privately matches places in your photos to a global index Apple maintains on our servers. We apply homomorphic encryption and differential privacy, and use an OHTTP relay that hides IP address. This prevents Apple learning about the information in your photos. You can turn off Enhanced Visual Search at any time on your iOS or iPadOS device by going to Settings > Apps > Photos. On Mac, open Photos and go to Settings > General.”
I agree with you 100.
I think Apple is doing a couple things fora couple different reasons (all of which I bother me) yet some worse than others
It's because it's a Triple (or more) Monetary Win Business Strategy. I'll explain.
Has anyone noticed how Apple keeps telling you your storage is "almost full" then wants you to upgrade to ICloud+? Because After selling that expensive phone they don't want to wai to squeeze more money out of you. They set ALZl defaults to open and upload to iCloud, so if you don't change them you are Icloudkng EVERTHING. Which fills up. Plus certain things you can't NOT iCloud bc you don't have a choice. What takes the MOST space and is most important to most people? Photos/Vi. So -) they auto default to upload/iCloud all plus your photos. Then tell you you are out of space and ask you to purchase more and more storage for cost each month from $1-$10.
But - ever tried to DOswLOAD your photos back to phone, and downgrade your storage? You can't. I'm almost impossible. The various instructions online don't work. You can do it with dedktop or laptop but even then they will try to reload and further even THEN there are ALZl kinds of bugs and glitches to getting photos back locally only.
So first part - they set all to upload and make local only imposssible and downgrading storage impossible. I had to take screenshots and file fraud claims to get them to downgrade m and stop charging me 9.99 and 1.99 each month.
But even then.
I'm pretty sure they ARE keeping all those photos we supposedly download back OFF iCloud and don't want their storage for AND of course everyone else photos and videos--Why? Second part of strategy (my opinion) - bc they are using ALL our collective photos and videos to Train Large AI models and computers and to run simulations, to prepare for future robots that are more like like, etc. yes U sound a bit conspiracy theorish - but it IS what they are doing! Plus aggregating all the data itself within our devices in photos/videos and other meta data - they can also run that through and sell it, store, make profiles on us, profilers on regions and correlations, etc - all of which is sellable now to various companies AND will be sellable in new ways in the future.
So they benefit triple on us.
I think there should be federal laws saying -1) All settings must be defaulted to highest privacy (ie closed) and 2) requiring Intentional option in, 3) Plain explanations of what "features and settings do, or how could be used, with clear warnings and subject to Lawsuits for fraudulent inducement and invasion of privacy if those descriptions are understandable by an ordinary reasonable person - NOT an ordinary TECHNICAL engineer. If they can't explain what it does to an ordinary person, it probably is doing stuff that ordinary person WOULD not consent to.
And 4 and 5) of new federal laws I wish there - 4) they can't reset or change your privacy settings to make them MORE open after each update, and again req opt-in not opt-out, 5) anyone who does Opt-In - should get metrics of all data disclosed - with a price per amount and data type - PAID back to them. So for example - there should some Global Standards of Data Types - Levels - some more valuable than
Others. EXAMPLE/Hypo If it collect "100MBs/amount" of Valuable Level 1 Data, which is Valuaed at $.005/MB then it owes you 100x.005= 5cents. And so on.
There should be classifications of data, with Intl standards on the Value of that DATA, and if you did OPT-IN, you get an accounting each month of what data and how much of it did THEY collect, and theyOWE you for collecting it.
Bc let's face it - how are they building the powerful computers and AI things they will trillions on? It's on us. Our data. How are making billions in sass and sales- on us, our data.
As it stands NOW they ought to pay us even for data we don't opt-in to but try to
Opt out of and can't!! And for all past data they've collected. And for my own time, and everyone else's time - Each month when a new update comes out - THAT EACH TIME resets all those settings I spent hours to reset back to close - and to research those hundreds of "Feature Flags" to find out which are good, which probably are NOT good and which Are in between or who know so roll the dice either way.
I set all settings to closed bc they auto default to everything open, everything sharing, Bluetooth on, public WiFi on, etc everything "uploading" to iCloud. It takes me HOURs. The Accessibility commands and options alone take me an hour to reset. Yet - even when it's turned off - sometimes my phone turns back on something even without updates bc of bugs of hidden features Apple isn't disclosing. Yet after each update all those settings I spent hours on on phones and Mac and iPad, etc- they default BACK to open. And I spend hours again. And it's taking longer each time, and updates while good for other things- for this it sucks, and updates are WAY more frequent. Makes me crazy.
I'm not a hacker or a tech like y'all probably are. So I apologize for my long rant by a novice such as myself. I'm just a normal person who likes to read and research and teach myself things so even though I'm not technical I doread a lot and so I'm not an idiot either. I've been a Mac user for over 20 years. Use to an @mac, then @me, and of course now @icl - things starting really going to shit and having sync/corruption issues and bugs, etc in 2019, got way worse in 2021, and now it's all the time. It's so frustrating though bc everything - is Mac-it's the invested infrastructure of our lives - to use an analogy:
it's like Apple/Icloud is a Hotel California. It's used to be a nice hotel though. Now it's a Zombie Apocalypse Hotel California. But there aren't any other "hotels" out there that are any better either I'm afraid.
What “places” would Apple be able to match that aren’t either obvious or irrelevant? (e.g. “The Grand Canyon” is obvious. The interior of a home is irrelevant.) Maybe the interior of a restaurant?
I hate to jump to conspiracy theories, but the mechanism seems like a more effective version of their prior CSAM scanning scheme[1]. Best part? It doesn’t even require iCloud usage.
Oh I dunno. Your comment made me chuckle and I’m largely on the fence on this one but it doesn’t seem too outrageous to say that maybe this is a slight invasion of people’s privacy without asking them to opt in.
You're kidding yourself if you think apple is in the business of protecting your data beyond marketing itself as such. Just like all it's competition it's in the business of using your data like them.
> You're kidding yourself if you think apple is in the business of protecting your data beyond marketing itself as such.
It is entirely possible to create a business where others ignore user demands, like, say, privacy. Apple does not have a massive advertising incentive to process user data as others do. I am not a huge Apple fan (typing this from my Windows machine), but I am not ashamed to admit that some companies are good at some things, and others at other things (insane idea I know)
Apple does not have to homomorphically encrypt your data. They do so because they have "made privacy a feature," as they say in their marketing. You can also just read the research papers or code if you want, since they have published those, too. Have you read those?
I am merely comparing homomorphic encryption, client-side vectorization and ML, and so on, to not doing those things. Nothing to do with marketing. Read the manual
Apple doing the best in class job for privacy and you guys would rather go after it vs google, meta and others who have significantly much massive issues?
I wonder how much of the people that are being upset about an option that recognizes landmarks and that is build in a privacy minded way are using chatGTP, windows, google or socials
The writer of the post admits that he doesn’t understand the very tech lite explanation of Apple nor read about Apple AI in general and the way it was setup. A lot of people are upset that (maybe, but unlikely) Apple knows that you made the 10 billionth photo of the Golden Gate Bridge. But continu using all sorts of services, living in sec camera heavy cities etc.
Not an apple fan but from all big tech corporations they have the least amount of interest in mining user data from a business perspective
>>Not an apple fan but from all big tech corporations they have the least amount of interest in mining user data from a business perspective
I think it's more that they realised they couldn't beat google at that game and hoped to see personal or governmental push back like GPDR in the EU that would then give them an advantage. Which would then give them an advantage. The day they stop having that belief they will invade your privacy.
What I want is very simple: I want software that doesn't send anything to the Internet without some explicit intent first. All of that work to try to make this feature plausibly private is cool engineering work, and there's absolutely nothing wrong with implementing a feature like this, but it should absolutely be opt-in.
Trust in software will continue to erode until software stops treating end users and their data and resources (e.g. network connections) as the vendor's own playground. Local on-device data shouldn't be leaking out of radio interfaces unexpectedly, period. There should be a user intent tied to any feature where local data is sent out to the network.
So why didn't Apple just simply ask for user permission to enable this feature? My cynical opinion is because Apple knows some portion of users would instantly disallow this if prompted, but they feel they know better than those users. I don't like this attitude, and I suspect it is the same reason why there is an increasing discontent growing towards opt-out telemetry, too.
This mindset is how we got those awful cookie banners.
Even more dialogs that most users will blindly tap "Allow" to will not fix the problem.
Society has collectively decided (spiritually) that it is ok signing over data access rights to third parties. Adding friction to this punishes 98% of people in service of the 2% who aren't going to use these services anyway.
Sure, a more educated populous might tip the scales. But it's not reality, and the best UX reflects reality.
Nope, collective indifference to subpar user experiences has gotten us those lousy cookie banners.
Web sites could legally use cookies for non-tracking purposes without cookie banners but considering people have not stopped visiting sites despite the fugly click-through cookie banners makes them a failure.
All it takes is for 50% of the internet users to stop visiting web sites with them, and web site authors will stop tracking users with external cookies.
18 replies →
No. A significant number of people care about Privacy which is why 1. Apply was targeting them with Ads and 2. AdBlock did hurt Google's business. Also care is different from go to war (as in install Linux and manually setup a privacy shield + Tor + only transact in Monero). Some people do that out of principal. Many people want the Privacy features but with the ease of use.
2 replies →
> This mindset is how we got those awful cookie banners.
The only thing I've found awful is the mindset of the people implementing the banners.
That you feel frustration over that every company has a cookie banner, is exactly the goal. The companies could decide that it isn't worth frustrating the user over something trivial like website analytics, as they could get that without having to show a cookie banner at all.
But no, they want all the data, even though they most likely don't use all of it, and therefore are forced to show the cookie banner.
Then you as a user see that banner, and instead of thinking "What a shitty company that don't even do the minimal work to not having to show me the cookie banner", you end up thinking "What a bad law forcing the company to inform me about what they do with my data". Sounds so backwards, but you're not the first with this sentiment, so the PR departments of the companies seems like they've succeed in re-pointing the blame...
16 replies →
The cookie banners are an example of malicious compliance.
1 reply →
Actually, if my mindset were leading, we wouldn't have cookie consent banners because we would've just banned non-essential tracking altogether.
3 replies →
With cookie banners, legislation said that every website needed to ask for consent -- a thousand sites, a thousand banners.
Operating system level controls, though, provide a single control plane. One can very easily imagine OS-level toggles per application of:
[No Internet, No Internet outside your own app-sandbox, Ask me every time, Everything is allowed].
No opt in from apps required -- they might break if the network is disabled, but the user is still in control of their data.
I think society has collectively "decided" in the same way they "decided" smoking in a restaurant is great.
There's little to no conscious choice in this. But there is a lot of money in this. Like... a LOT of money. If I were to try to influence society to be okay with it, it would be a no brainer.
So, to me, it's obvious that society has been brainwashed and propagandized to accept it. But doing so generates hundreds of billions if not trillions of dollars. How, exactly, such manipulation is done is unknown to me. Probably meticulously, over the course of decades if not centuries. I know that the concept of privacy during the writing of the constitution was much, much more stringent than it was in the 70s, which is much more stringent than it is today.
But, I am very confident it is happening.
1 reply →
I think it's clear that users should be able to have their own agents that make these decisions. If you want an agent that always defers to you and asks about Internet access, great. If you want one that accepts it all great. If you want one that uses some fancy logic, great.
u-Block Origin's annoyances filters take care of the cookie banners, giving the best of both worlds: no banners and a minimal amount of tracking.
(The "I don't care about cookies" extension is similarly effective, but since I'm already running u-block origin, it makes more sense to me to enable it's filter.)
2 replies →
Why does it have to be more friction?
Users had a global way to signal “do not track me” in their browser. I don’t know why regulators didn’t mandate respecting that instead of cookie consent popups.
Apple IDs could easily have global settings about what you are comfortable with, and then have their apps respect them.
I’m spitballing here but wouldn’t another way to handle it would be to return dummy / null responses by redirecting telemetry calls to something that will do so?
This would have the added benefit of being configurable and work on a bunch of apps instead of just one at a time too
I use Firefox focus on android and Firefox with ubo and others..
On desktop and Firefox app, I only browse through private browsing so cookies are mostly irrelevant as session ends as soon as all windows close.
1 reply →
I always click disallow.
And if you design software that uses tracking and what not. Go fuck yourself.
Better to have cookie banners than made-up 'collective decision'.
This string of unprovable assumptions is not how you do things.
[dead]
[flagged]
Not really. A mandatory opt-in option at the browser level would be the correct way to do it, but legislation forced instead those cookie banners onto the webpage.
1 reply →
Opt in doesn't work, it never did.
The vast majority (>95%) of users does not understand what those pop-ups say, seems fundamentally incapable of reading them, and either always accepts, always rejects, or always clicks the more visually-appealing button.
Try observing a family member who is not in tech and not in the professional managerial class, and ask them what pop-up they just dismissed and why. It's one of the best lessons in the interactions between tech and privacy you can get.
Well, then >95% of users won't be using $FEATURE. Simple as that. The fact that users for some reason no not consent to $FEATURE the way corporations/shareholders would want them to does not give anyone the right to stop asking for consent in the first place.
1 reply →
When looked at from another angle, opt-in does work.
By adding that extra step forcing users to be aware of (and optionally decline) the vendors collection of personal data, it adds a disincentive for collecting the data in the first place.
In other words, opt-in can be thought of as a way to encourage vendors to change their behaviour. Consumers who don't see an opt-in will eventually know that the vendor isn't collecting their information compared to others and trust the product more.
As much as I hate cookie consent dialogs everywhere, the fact is that it is clearly working. Some companies are going as far as to force users to pay money in order to be able to opt out of data collection. If it wasn't so cumbersome to opt-out, I reckon the numbers for opt-out would be even higher. And if companies weren't so concerned about the small portion of users that opt-out, they wouldn't have invested in finding so many different dark patterns to make it hard.
It is definitely true that most users don't know what they're opting out of, they just understand that they have basically nothing to gain anyway, so why opt-in?
But actually, that's totally fine and working as intended. To be fair to the end user, Apple has done something extremely complicated here, and it's going to be extremely hard for anyone except for an expert to understand it. A privacy-conscious user could make the best call by just opting out of any of these features. An everyday user might simply choose to not opt-in because they don't really care about the feature in the first place: I suspect that's the real reason why many people opt-out in the first place, you don't need to understand privacy risks to know you don't give a shit about the feature anyway.
Opt in works!
If you do not want it (and that is >90% of people, who never asked for it, never requested it, but was forced upon them these 'enriched' lies and exposure to corporate greed).
> Try observing a family member who is not in tech
This is everyone, it is universal, I've met many people "in tech" who also click the most "visually appealing" button because they are trying to dismiss everything in their way to get to the action they are trying to complete.
The microcosm that is HN users might not just dismiss things at the 95%+ rate, but that is because we are fed, every day, how our data is being misappropriated ate every level. I think outside of these tiny communities, even people in tech, are just clicking the pretty button and making the dialog go away.
The issue really isn't opt-in itself but how the option is presented.
I agree that a lot of people don't read, or attempt to understand the UI being presented to them in any meaningful manner. It really is frustrating seeing that happen.
But, think about the "colorful" option you briefly mentioned. Dark patterns have promoted this kind of behaviour from popups. The whole interaction pattern has been forever tainted. You need to present it in another way.
Informed consent is sexy. In the Apple ecosystem, we’re literally paying customers. This is ridiculous. This line you parroted is ridiculous. This needs to stop.
Opt in works great, pop-up dialogues do not.
[flagged]
8 replies →
> So why didn't Apple just simply ask for user permission to enable this feature?
That’s an interesting question. Something to consider, iOS photos has allowed you to search for photos using the address the photo was taken at. To do that requires the Photos app to take the lat/long of a photos location, and do a reverse-geo lookup to get a human understandable address. Something that pretty much always involves querying a global reverse-geo service.
Do you consider this feature to be a violation of your privacy, requiring an opt-in? If not, then how is a reverse-geo lookup service more private than a landmark lookup service?
> To do that requires the Photos app to take the lat/long of a photos location, and do a reverse-geo lookup to get a human understandable address.
It seems trivially possible to do this in a more privacy preserving way: geocode the search query and filter photos locally.
No idea how Apple implements it though.
It's a complete violation if it's a new or changed setting from the default state of the user not having it possible.
Something to consider - location is geo-encoded already into photos and doesn't need this uploaded to Apple servers. Searching can be done locally on device for location.
Apple goes as far as to offer a setting to allow the user to share photos and remove the geocoding from it.
Offering a new feature is opt-in.
Unfortunately, against my better wishes, this only erodes trust and confidence in Apple that if this is happening visibly, what could be happening that is unknown.
> Do you consider this feature to be a violation of your privacy, requiring an opt-in?
I suppose in some sense it is, as it a reverse-geo lookup service, but it's also no where near to the front in the location privacy war.
Cell phone providers basically know your exact position at all times when you have your phone on you, credit card companies know basically everything, cars track driving directly, etc. etc.
I can see why some people would be up in arms but for me this one doesn't feel like missing the forest for the trees, it feels like missing the forest for the leaves.
1 reply →
You’re right. But: Anyone in IT or tech, thinking deeply about the raw facts. They know it always boils down to trust, not technology.
The interesting thing is that Apple has created a cathedral of seemingly objective sexy technical details that feel like security. But since it’s all trust, feelings matter!
So my answer is, if it feels like a privacy violation, it is. Your technical comparison will be more persuasive if you presented it in Computer Modern in a white paper, or if you are an important Substack author or reply guy, or maybe take a cue from the shawarma guy on Valencia Street and do a hunger strike while comparing two ways to get location info.
4 replies →
This would work only if you've already given the Camera app permission to geotag your photos, which I haven't, so it may be a nonissue.
12 replies →
And the result is https://chromewebstore.google.com/detail/i-still-dont-care-a...
Personally I do not believe these popups serve any purpose, because I ultimately cannot (at least in a reasonable way) prove that the website is acting in good faith. Asking me whether the app should phone home doesn't really guarantee me pressing "no" will actually prevent the tracking.
I am continuously surprised at how we convince ourselves privacy at scale will work with a varying amount of yes/no buttons. There are 2 ways to trust software 1. be naive and check whether "privacy first" is written somewhere 2. understand the software you are running, down to the instructions it is able to execute.
The permission popups also lack granularity. When giving access to my contact list, which contacts does it actually access? Can I only give access to contacts name and not phone numbers? Is it for offline or online processing? If online, should we have another popup for internet access? But then, can I filter what kind of internet stuff it does? You go down the rabbit hole and eventually end up with a turing-complete permission system, and if you don't, your "privacy" will have some hole to it.
Even with opt-in a vendor will keep harassing the user until they tap "yes" in an inattentive moment.
And I've been in situations where I noticed a box was checked that I'm sure I didn't check. I want to turn these things off and throw away the key. But of course the vendor will never allow me to. Therefore I use Linux.
I want to turn these things off and throw away the key. But of course the vendor will never allow me to. Therefore I use Linux.
I hate to break it to you, but these things happen in Linux, too.
It's not the operating system that's the problem. It's that the tech industry has normalized greed.
4 replies →
For what it's worth, I use Linux, too, but as far as phones go, stock phones that run Linux suffer from too many reliability and stability issues for me to daily drive them. I actually did try. So, as far as phones go, I'm stuck with the Android/iOS duopoly like anyone else.
1 reply →
> I want software that doesn't send anything to the Internet without some explicit intent first
I want this too, but when even the two most popular base OSes don't adhere to this, I feel like it's an impossible uphill battle to want the software running on those platforms to behave like that.
"Local-first" just isn't in their vocabulary or best-interest, considering the environment they act in today, sadly.
Developers of software want, and feel entitled to, the data on your computer, both about your usage within the app, as well as things you do outside of the app (such as where you go and what you buy).
Software will continue to spy on people so long as it is not technically prohibited or banned.
I don’t.
I highly suggest everyone else does their darnedest not too either. Don’t do it in your own software. Refuse and push back against it at $dayJob.
I realize that my small contribution as a privacy and data-respecting SWE is extremely small, but if we all push back against the MBAs telling us to do these things, the world will be better off.
15 replies →
In the OP article it seems more like users demand to search their photos by text, and Apple has put in a huge effort to enable that without gaining access to your photos.
1 reply →
Years ago I developed for iOS as an employee. In my case, it was the product managers that wanted the data. I saw it as a pattern and I hated it. I made my plans to leave that space.
Only recently. If anyone's grown up with a world only knowing this, it might be part of why it might not stand out as much.
> So why didn't Apple just simply ask for user permission to enable this feature? My cynical opinion is because Apple knows some portion of users would instantly disallow this if prompted, but they feel they know better than those users. I don't like this attitude, and I suspect it is the same reason why there is an increasing discontent growing towards opt-out telemetry, too.
I'm just not sure why Apple needed to activate this by default, other than not draw attention to it... and doing so that was more important than the user's rights to the privacy they believe they are purchasing on their device.
I don't care what convenience i'm being offered or sold. If the user has decided what they want and the premium they are paying for Apple, it must be respected.
This makes me wonder if there is an app that can monitor all settings in an iPhone both for changes between updates, and also new features being set by default to be enabled that compromise the user's known wishes.
All this AI, and this is still overlooked.
I'm hoping it was an oversight.
Consent for complex issues is a cop out for addressing privacy concerns. Users will accept or reject these things without any understanding of what they are doing either way. Apple seems to have taken a middle ground where they de-risked the process and made it a default.
This is a “look at me, Apple bad” story that harvests attention. It sets the premise that this is an unknown and undocumented process, then proceeds to explain it from Apple documentation and published papers.
"What I want is very simple: I want software that doesn't send anything to the Internet without some explicit intent first."
It exists. I use such software everyday. For example, I am submitting this comment using a text-only browser that does not auto-load resources.
But this type of smaller, simpler software is not popular.
For example, everyone commenting in this thread is likely using a browser that auto-loads resources to submit their comments. HN is more or less a text-only website and this "feature" is not technically necessary for submitting comments. All so-called "modern" web browsers send requests to the internet without explicit intent first. IN addition to auto-loading resources, these browsers automatically run Javascript which often sends further requests never intended by the web user.
Brand new Apple computers now send packets to the internet as soon as the owner plugs them in for the first time. This may enable tracking and/or data collection. Apple proponents would likely argue "convenience" is the goal. This might be true. But the goal is not the issue. The issue is how much the computer owner is allowed to control the computer they buy. Some owners might prefer that the computer should not automatically send packets to remote Apple servers. Often it is not even possible to disable this behaviour. Computer purchasers never asked for these "convenenience" features. Like the subject of this submission, Apple Photos, these are Apple's decisions. The computer owner is not allowed to make decisions about whether to enable or disable "convenience" features.
As the court acknowledged in its opinion in US v Google, default settings are significant. In this case, it is more than a default setting. It is something the owner cannot change.
>I want software that doesn't send anything to the Internet without some explicit intent first.
I too want exactly that, which got me thinking, that's what firewalls are for! DROP OUTBOUND by default, explicit allow per-app.
On Andoid, iptables-based firewalls require root, which wasn't a good option for me (no twrp support for my device), so after some searching I stumbled upon NetGuard - open source and rootless, implements a firewall using Android's VPN service (you can configure Android to route all traffic through this "VPN" which is actually a local firewall). The downside is you can't use an actual VPN (except with some complicated setup involving work profiles and other apps). I've been using it for a couple of weeks and am very satisfied, I noticed apps phoning home which I did not want to, like a scanning app I had used to scan private documents in the past, perhaps an oversight on my part.
Use a rooted Android phone with AFWall+ installed, with default block rules. Even just LineageOS allows you to set granular network settings per app, though it's not preemptive like AFWall.
Can't run various banking apps and can't run PagerDuty on a rooted device due to Google Play API Integrity Check. The ecosystem is closing in on any options to not send telemetry, and Google is leading the way in the restrictions on Freedom.
6 replies →
“Use a rooted…”
Aaaaand no.
2 replies →
IMO, there should be 3 categories of users, and they can choose a system wide setting that applies across all their apps and settings:
* Bulletproof
* Privacy Conscious
* Normal (recommended)
That way users are roughly opting in and opting out in a way that aligns with their desires
> Trust in software will continue to erode
> there is an increasing discontent growing towards opt-out telemetry
Really? That's news to me. What I observed is people giving up more and more privacy every year (or "delegating" their privacy to tech giants).
Absolutely! The important bit is that users have no choice in the matter. They're pushed into agreeing to whatever ToS and updating to whatever software version.
The backlash against Microsoft's Windows Recall should serve as a good indicator of just how deeply people have grown to distrust tech companies. But Microsoft can keep turning the screws, and don't you know it, a couple years from now everyone will be running Windows 11 anyways.
It's the same for Android. If you really want your Android phone to be truly private, you can root it and flash a custom ROM with microG and an application firewall. Sounds good! And now you've lost access to banking apps, NFC payments, games, and a myriad of other things, because your device no longer passes SafetyNet checks. You can play a cat-and-mouse game with breaking said checks, but the clock is ticking, as remote attestation will remove what remains of your agency as soon as possible. And all of that for a notably worse experience with less features and more problems.
(Sidenote: I think banking apps requiring SafetyNet passing is the dumbest thing on planet earth. You guys know I can just sign into the website with my mobile browser anyways, right? You aren't winning anything here.)
But most users are never going to do that. Most users will boot into their stock ROM, where data is siphoned by default and you have to agree to more data siphoning to use basic features. Every year, users will continue to give up every last bit of agency and privacy so as long as tech companies are allowed to continue to take it.
21 replies →
One of the reasons is because telemetry and backdoors are invisible. If the phone was showing a message like "sending your data to Cupertino" then users were better aware of this. Sadly I doubt there will be a legal requirement to do this.
1 reply →
Apple seems to be the best option here too. They seem to have put in a huge effort to provide features people demand (searching by landmarks in this case) without having to share your private data.
It would have been so much easier for them to just send the whole photo as is to a server and process it remotely like Google does.
> What I observed is people giving up more and more privacy every year (or "delegating" their privacy to tech giants).
Are people giving up their privacy? Looks to me it’s being taken without consent, via enormous legalese and techniques of exhaustion.
1 reply →
Whether or not people in general are aware of this issue and care about it, I think it's pretty disingenuous to characterize people as willfully giving up their privacy because they own smartphone. When stuff like this is happening on both iOS and Android, it's not feasible to avoid this without just opting out of having a smartphone entirely, and representing as a binary choice of "choose privacy or choose not to care about privacy" is counterproductive, condescending, and a huge oversimplification.
1 reply →
Come on, being forced to give up privacy is eroding privacy and increasing discontent.
forced can also mean the whole no privacy by default and dark patterns everywhere.
Do you honestly believe people understand what they’re doing?
Nowhere in marketing materials or what passes for documentation on iOS we see an explanation of the risks and what it means for one’s identity to be sold off to data brokers. It’s all “our 950 partners to enhance your experience” bs.
1 reply →
The shorter answer is that it's your data, but it's their service. If you want privacy, you should use your own service.
And for how cheap and trivial syncing photos is, any mandatory or exclusive integration of services between app/platform/device vendors needs to be scrutinized heavily by the FTC.
> Trust in software will continue to erode until software stops treating end users and their data and resources (e.g. network connections) as the vendor's own playground. Local on-device data shouldn't be leaking out of radio interfaces unexpectedly, period. There should be a user intent tied to any feature where local data is sent out to the network.
I find that there is a specific niche group of people who care very much about these things. But the rest of the world doesn't. They don't want to care about all these little settings they're just "Oh cool it knows it's the Eiffel tower". The only people who are becoming distrusting of software are a specific niche group of people and I highly suspect they're going to be mad about something.
> So why didn't Apple just simply ask for user permission to enable this feature?
Because most people don't even care to look at the new features for a software update. And let's be serious that includes most of us here otherwise, this feature would have been obvious. So why create a feature that no one will use? It doesn't make sense. So you enable it for everyone and those who don't want it opt-out.
>> Trust in software will continue to erode until software stops treating end users and their data and resources
Trust in closed-source proprietary software. In other words: trust in corporate entities. Trust in open-source software is going strong.
Not a given though. Ubuntu phones home a lot by default.
Try disabling the motd stuff - it's quite pernicious by design.
And removing the ubuntu-advantage package disables the desktop. lol.
I want a hardware mic switch. We are an iHouse with one exception and that's a SheildTV that is currently out of order because I want to reset it and haven't found time in, oh..., weeks. Anyway, out of the blue one of the kids asked about Turkish delights and wonders where the name came from. SO and I facepalm then explain. Not an hour later she gets something in her Facebook feed: 15 interesting facts about Turkey.
This is just too much of a coincidence. I know, I know, this "... isn't Apple's fault" blah blah. Bullshit it's not. They can't have it both ways where they say their app store process is great and then they allow this shit.
So you don't want a browser?
A browser (without telemetry) is surely a good definition of something that doesn't initiate network calls before user intent
Browsing the Internet is explicit intent! Some of the stuff enabled by JavaScript definitely tows the line but at the very least that's not really the direct fault of the browser.
You're absolutely right! And the decision to make this opt-out feels dismissive
Most people nowadays use Web based apps, which don't even need to ask anything, who knows what server side is doing.
Which is kind of ironic in places like HN, where so many advocate for Chromebooks.
Your location data, encoded in photo you take with the phone's camera, being extracted by Apple is what this article is about.
How many people use a web based camera or web based photo album app?
2 replies →
Would you mind giving an example of something bad that could happen to somebody as a result of Apple sending this data to itself? Something concrete, where the harm would be realized, for example somebody being hurt physically, emotionally, psychologically, economically, etc
Once upon a time, I worked for a pretty big company (fortune 500ish) and had access to production data. When a colleague didn't show up at work as they were expected, I looked up their location in our tracking database. They were in the wrong country -- but I can't finish this story here.
Needless to say, if an Apple employee wanted to stalk someone (say an abusive partner, creep, whatever), the fact that this stuff phones home means that the employee can deduce where they are located. I've heard stories from the early days of Facebook about employees reading partner's Facebook messages, back before they took that kind of stuff seriously.
People work at these places, and not all people are good.
3 replies →
Easy, consider a parent taking pictures of their kid's genitals to send to their doctor to investigate a medical condition, the pictures getting flagged and reported to the authorities as being child pornography by an automated enforcement algorithm, leading to a 10-month criminal investigation of the parent. This exact thing happened with Google's algorithm using AI to hunt for CP[1], so it isn't hard to imagine that it could happen with Apple software, too.
[1] https://www.koffellaw.com/blog/google-ai-technology-flags-da...
1 reply →
Arrogant Apple always knows best! Which is why I've always said, and I'll continue saying, fuck Apple.
[flagged]
and there's absolutely nothing wrong with implementing a feature like this, but it should absolutely be opt-in
This feature is intended to spy on the user. Those kinds of features can't be opt-in. (And yeah, holomorophic "privacy preserving" encryption song-and-dance, I read about that when it came out, etc).
This is an incredibly shallow dismissal that states the opposite of Apple's claim with zero evidence or reasoning and hand-waves away the very real and well-researched field of homomorphic encryption.
Users of my (free, open-source) app seem surprised to learn that we've got zero insight into usage patterns. There are situations where a small amount of anonymous telemetry would be extremely helpful but I'm not going to touch it with a barge-pole.
Opt-in makes the data useless - not just in terms of the huge drop in quantity but because of the fact it introduces a huge bias in the data selected - the people that would opt-in are probably not a good sample of "typical users".
Opt-out - no matter what safeguards or assurances I could provide is unacceptable to a subset of users and they will forcefully communicate this to you.
Don't get me wrong - I understand both the ease at which bad actors abuse telemetry and the ease in which "anonymous data" can prove to be nothing of the kind in a multitude of surprising ways.
But it's hard not to feel a little sad in a "this is why we can't have nice things" kind of way.
I can't remember where I saw this before. However, there was a site that collected analytics data client side in a circular buffer (or something), and there was a menu in the settings to send it back one-time or always, or download it yourself. If you experienced an error, they would pop up in a toast to share the analytics data with them so they could help fix the problem. You could, of course, decline.
That was probably the best system I'd seen, but I can't remember what site it was.
On macos (maybe tiger or leopard era), apple used to pop up a crash dialog, with a "send to apple?" prompt. And you could say no.
they did away with that.
3 replies →
maybe it was offen? i remember seeing it a while ago but i don’t know of any sites using it. https://www.offen.dev/
1 reply →
I built the same for my browser extension (effectively dead product) -- would love to see if this pattern has a name so I can share it more widely!
Maybe the Datadog Flare works like this?
1 reply →
Consent is the key issue binding all. There is complete lack of consent when there is no opt-out and great degradation when the default is opt-out. Trust is the only means to consent.
1) Opt-in, Opt-survey, Opt-out is the only ternary to build trust. Survey is an active validator of trust and assists in low-bandwith communication. Question should be presented to the end user the first time using it or the next time the application starts and this feature was added.
2) Provide the exact analytical information you want to the end user so they can parse it too. The means to self-evaluate allowed information to be shared with providing the reports or views improves trust.
3) Known privilege to trust leads to more consent. Having priority support with features and bugs could be aligned with those that Opt-in. Analytical history / performance may assisting in solving the recent bug that was reporter.
Apple, Microsoft, Google, and all apply ambiguity to their analytical sharing without details, not how they use it and can abuse it. Most don't even provide an Opt-out. I don't trust these organizations but I must engage with them through my life. I don't have to use Facebook or Twitter and don't. I accept the Steam survey.
RFC with an agreed upon analytical standard could be step to solving the latch of analytical information the open source community would benefit from. Both parties consenting to agreed upon communication.
*My Point of View; meta data is still personal data. Without the user the data and the meta data would not existing. Since the end user is the entropy to meta data they own the meta and the data.
Yes - I understand but in many (or even most) cases, opt-in makes the data worthless. There's literally no point collecting it.
1 reply →
Would there be a way to do the stats gathering on device, then once every few months send a popup with statistics?
Not sure what bias it adds
Like
"hey, we make this app, and we care about privacy, here is the information we have gathered over your usage for the past month, can we send this to ourselves, so that we can use it to improve the app?"
And then show human readable form of what data was collected.
Just as a reference of existing implementations of this: This is essentially how Valve/Steam collects hardware details from users/clients. Every now and then, a popup appears asking the user if they'd like to participate in the "Hardware Survey", together with all the data that would be submitted if they accept.
Seems to me like a great implementation.
The podcast app I use, AntennaPod (far better for me than other apps, available on F-Droid, no affiliation!) just gave me a local-only year in review. I thought it was a great touch, and would be happy to have then shared the data from that with the app's makers.
You'd still have extremely biased data - people who blindly click OK on every pop up are not representative of your typical user; people who get nightmares after hearing the word "telemetry" and will gather the pitchforks if they hear any hint of will always refuse, but depending on your app, might be your typical user (e.g. for self-hosted picture sync and catalogue, who is the target audience - people who don't trust Apple/Google/Amazon/Dropbox to store their images privately)
2 replies →
This sort of sounds like the Steam Hardware Survey. They do not collect the data willy-nilly, they ask you every few months if you want to participate in a one-time check.
I have an incentive to see if the Linux desktop share has increased, so I usually run the survey for them to get my data point in. I also suppose the "gamer" crowed likes to show off how powerful their "rig" is, so I would imagine they commonly also run the survey for that reason as well.
1 reply →
> Opt-in makes the data useless - not just in terms of the huge drop in quantity but because of the fact it introduces a huge bias in the data selected - the people that would opt-in are probably not a good sample of "typical users".
Why? I don't think that's obvious. It may also be related to the way the opt-in is presented. In general, I would expect this to be a workable solution. Even if the opt-in group deviates from the "typical user", it's the best data you can get in an honest and ethically sound way. This should certainly be better than no data at all?
For any website/app that presents an opt-in cookie consent banner this is implicitly already the case.
Yes, this is one of the main reasons people mostly build on web. It's very difficult to make desktop software better, and especially Linux users are hostile to patterns that would make improvements possible
>Opt-in makes the data useless
Hardly. It just has some issues with regards to what you also pointed out, bias for one. But it still provides valuable insight into usage patterns, systemic issues, and enables tracking effects of developments over time. Correcting the bias is not a bigger task than it is now - I'm sure you already have an idea about feedback to different features according to reviews, user reports, discussions, and so on. Opt-in is the same, just much better.
Maybe the solution lies in radical transparency: explaining exactly how and why telemetry would help, then letting users decide. But even that requires trust...
Is there a Github API for creating issues? I also maintain a free, open-source app and would love to make it easy for a crash to give users a button that opens a Github issues form--allowing users to see what crash data is populated and submit it if they want.
Data collection and telemetry is sadly lemon market type of situation. The most trustworthy developers are precisely the ones who don't collect data from users
This can only ever be opt-in if you want to stay on the legal side of the GDPR (and equivalents in other jurisdictions). You can ask, but the default needs to be "no" if no answer is given.
I provide telemetry data to KDE, because they default to collecting none, and KDE is an open-source and transparent project that I'd like to help if I can. If I used your app, I would be likely to click yes, since it's open-source. Part of the problem I have with projects collecting user data is the dark patterns used or the illegal opt-out mechanism, which will make me decline sending telemetry every time, or even make me ditch it for an alternative. An app that asks:
...with equal weight given to both options, is much more likely to have me click Yes if none of the buttons are big and blue whilst the other choice is in a smaller font and "tucked away" underneath the other (or worse, in a corner or hidden behind a sub-menu).
Plus, I would think that SOME data would be better than NO data, even if there's an inherent bias leaning towards privacy-minded/power users.
> This can only ever be opt-in if you want to stay on the legal side of the GDPR
The GDPR only applies to personal data. You can collect things like performance data without opt-in (or even an opt-out option) as long as you are careful to not collect any data that can be used to identify an individual, so no unique device IDs or anything like that. Of course, you should be transparent about what you collect. You also have to be careful about combinations of data points that may be innocuous on their own but can be used to identify a person when combined with other data points.
Completely, 100% agreed:
> the only way to guarantee computing privacy is to not send data off the device.
> It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations. [...] By enabling the "feature" without asking, Apple disrespects users and their preferences. I never wanted my iPhone to phone home to Apple.
Regardless of how obfuscated or "secure" or otherwise "privacy-protecting" the feature is, the fact is that some information derived from one's personal content is transmitted, without prior consent. Even if the information is protected, all network queries are information. A timestamp that proves you took a certain action at a certain time (like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo), from a certain location (by correlating your location information at that time), etc etc.. and that's just the tip of the iceberg. Transmitting information from a user's device without their explicit consent is a violation of their privacy.
So Signal messages aren't secure because they're transmitted and so their "obfuscation" isn't enough to protect your data? Have you read what the author cited (and then admitted to not understanding) what Apple says they actually do to the data before transmission?
I could see an argument in the metadata (though there are multiple assumptions involved there, not least that they don't truly do OHTTP but instead conspire to learn at what timestamp a user took a picture), but if you already don't trust in what is essentially math, I'm not sure where the uncertainty and doubt ends
The difference being that the signal message is sent with consent: You literally press a button to send it there is a clear causal relationship between clicking the button and the message being sent.
The obvious difference is that by sending your photos with Signal, you are doing it willingly. You let it encrypt and decrypt willingly. You decide who gets it.
Here, Apple does that for you.
5 replies →
> like taking a photo, assuming stuff is sent to this service immediately upon adding a new photo
So you jumped to a conclusion based on an incorrect premise. This is easy to see that this does not happen immediately after taking a photo. One the network traffic will show this (and it won’t), two homomorphic encryption is expensive so it cannot. Photos classically doesn’t sync on demand, as most iPhone users will know by way if it telling you this in the photos app when it does sync. Most expensive operations are queued up for when the device is plugged in (and on WiFi) because it’ll otherwise drain battery.
you're splitting a very fine hair while ignoring the larger privacy implication of the feature. So the timestamp might or might not be delayed a bit from being perfectly accurate? So what? It still is data approximating when the photo was taken, even if the resolution were as bad as "within a few days"
1 reply →
How much size would it take to store a model of every known location in the world and common things?
For ex: I sent a friend a photo of my puppy in the bathtub and her Airpods (via iphone) announced "(name) sent you a photo of a dog in a bathtub". She thought it was really cool and so do I personally. That's a useful feature. IDK how much that requires going off-device though.
> That's a useful feature.
I’m really curious how this feature is considered useful. It’s cool, but can’t you just open the photo to view it?
6 replies →
I’m not an expert, but I would say extremely small.
For comparison Hunyuan video encodes a shit-ton of videos and rudimentary real world physics understanding, at very high quality in only 13B parameters. LLAMA 3.3 encodes a good chunk of all the knowledge available to humanity in only 70B parameters. And this is only considering open source models, the closed source one may be even more efficient.
1 reply →
These issues are all addressed in the Apple blog post that talks about how this feature is implemented. Two steps are taken to deal with these risks:
1) iOS creates additional fake queries, and all queries pass through scheduler that ensures you can use time-of-lookup to either discriminate real queries from fake queries, or identify when a photo was taken.
2) All queries are performed anonymously, with the use of a third party relaying service. So there’s no way for Apple to tie a specific query back to a specific device, or even IP address.
Between those two mitigating features. Getting hold of an individuals personal data using this feature requires you to first compromise the targets phone, to disable the fake queries. Then compromise the relaying party to correlate queries back to a specific IP address.
If you can manage all that, then quite frankly you’re a fool for expending all that effort. When you could just use your iOS compromise to have the device send you its location data directly. No need to faff about waiting for your target to take photos, then track multiple landmark lookups, carefully collecting a few bits of additional data per query, until you finally have enough to identify the location of your target or targets.
The whole thing reminds me of XKCD 538.
https://machinelearning.apple.com/research/homomorphic-encry...
Is there a way to verify the claims of obfuscation, security and privacy? Or is the only verifiable fact the sending of unknown data to apple by the photos app?
Is this just a smokescreen around slowly sneaking CSAM scanning back in after the pushback last time? The "default on" behavior is suspect.
[1] https://www.wired.com/story/apple-photo-scanning-csam-commun...
My thoughts exactly: "we've got this crafty image fingerprinting, the CSAM detection use proved too controversial to roll out, but let's get the core flows into something that sounds useful for users, so the code atays alive, improving, & ready for future expansion."
Whether such fingerprinting can reliably be limited to public "landmarks" is an interesting question, dependent on unclear implementation details.
Even if the user-visible search is limited to 'landmarks', does the process pre-create (even if only on-device) fingerprints of many other things as well? If so, it suddenly becomes possible for briefly-active non-persistent malware to instantly find images of interest without the wider access & additional processing it'd otherwise take.
> let's get the core flows into something that sounds useful for users
is it even that?
I don't see the benefit of this whatsoever
1 reply →
Yup, this is their way of injecting the "phone home" element via an innocuous rationale, "location matching". The global index will of course also match against other markers they deem worthy of matching, even if they don't return that to the user.
But wouldn't the homomorphic encryption prevent Apple's servers from knowing if there was a match or not?
11 replies →
Honestly, why the hell would Apple bother with such a contrived and machiavellian strategy to spy on their users?
They literally own the code to iOS. If they wanted to covertly track their customers, they could just have their devices phone home with whatever data they wanted to collect. Realistically there would be no way to know if this was actually happening, because modern devices emit so much encrypted data anyway, it wouldn’t be hard to hide some nefarious in all the noise.
Time Cook isn’t some Bond villain, sitting in a giant chair, stroking a white cat, plotting to take over the world by lulling everyone into a false sense of privacy (I mean Zuckerburg already did that). Apple is just a large giant corporation that wants to make money, and is pretty damn open about that fact. They clearly think that they can make more money by doubling down on more privacy, but that doesn’t work if you don’t actually provide the privacy, because ultimately, people are really crap at keeping secrets, especially when a media group would happily pay for a story, even at Apple.
1 reply →
Exactly like how Microsoft "backed off" Recall. Uuuuuntil they shoved it back in and made it undeleteable.
By removing it from the market, making enormous technical tweaks based on user feedback, and then putting it back on the market.
Yes my thoughts as well. The tech was so expensive I guess that they had a need to test / run it to proof it’s private? I mean the model to find landmarks in your photos could run locally as well or? Ok I’m not 100% sure here.
I assume that the model couldn’t run locally for some reason. Probably either uses too much power or needs too much memory.
No, it is not. Whatever their other failings, Apple doesn’t think that way.
The cynical reason: consider that you can’t plan engineering features of this scale without written documentation, which will always surface in court.
The prima facie reason: Apple genuinely wants to provide useful features requiring server participation.
This is incredibly naive and curiously defensive.
If this was a feature on its own then it would not be popular.
Citing national security, some danger will justify its existence.
Apple alone does not control and dictate what goes in, once you reach their level of size and wealth that exceed even developed countries, you ultimately cannot be the controller of your destiny purely as a profit orientated corporation.
ex) Meta, Microsoft, Google
Very likely yes. Why else would they add a feature that incurs costs for them as an update, at no cost to the users (and not even make a fuss about it)?
It is obvious they are monetizing this feature somehow. Could be as innocuous as them training their AI dataset, or feeding into their growing ad business (locations and other things identified in the photos), or collaboration with law enforcement for various purposes (such as notifying the CCP about people's Winnie-the-Pooh memes), or a lot more ominous things.
> Very likely yes. Why else would they add a feature that incurs costs for them as an update, at no cost to the users (and not even make a fuss about it)?
Erm, you’re aware of the whole Apple intelligence thing right? An entire product that costs Apple money, provided at “no cost” to the user (if you had an iPhone 15). Also every feature in an OS update has a costs associated with it, and iOS updates have cost money for the best part of a decade now.
Has it occurred to you that reason Apple includes new features in their updates is to provide customers with more reasons to buy more iPhones? Just because feature are provided at “no cost” at point of consumption, doesn’t mean Apple won’t make money in the long run, and selling user data isn’t the only way to monetise these features. Companies have been giving out “freebies” for centuries before the internet existed, and the possibility of large scale data collection and trading was even imaginable.
1 reply →
That whole incident was so misinformed.
CSAM scanning takes place on the cloud with all the major players. It only has hashes for the worst of the worst stuff out there.
What Apple (and others do) is allow the file to be scanned unencrypted on the server.
What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match.
That file in question would be able to be decrypted on the server and checked by a human. For everything else it was encrypted in a way it cannot be looked at.
If you had icloud disabled it could do nothing.
The intent was to protect data, children and reduce the amount of processing done on the server end to analyse everything.
Everyone lost their mind yet it was clearly laid out in the papers Apple released on it.
Apple sells their products in oppressive regimes which force them to implement region specific features. E.g. China has their own iCloud, presumeably so it can be easily snooped on.
If they were to add this anti-CSAM feature, it is not unreasonable to think that Apple would be forced to add non-CSAM stuff to the database in these countries, e.g. anything against a local dictatorship/ etc. Adding the feature would only catch the low hanging CSAM fruit, at the cost of great privacy and probably human life. If it was going to stop CSAM once and for all, it could possibly be justified, but that's not the case.
2 replies →
"It only has hashes for the worst of the worst stuff out there." [citation needed]
I know someone whose MS account was permabanned because they had photos of their own kid in the bathtub. I mean, I guess the person could have been lying, but I doubt they would even have been talking about it if the truth was less innocuous.
5 replies →
I can't believe how uninformed, angry, and still willing to argue about it people were over this. The whole point was a very reasonable compromise between a legal requirement to scan photos and keeping photos end-to-end encrypted for the user. You can say the scanning requirement is wrong, there's plenty of arguments for that. But Apple went so above and beyond to try to keep photo content private and provide E2E encryption while still trying to follow the spirit of the law. No other big tech company even bothers, and somehow Apple is the outrage target.
6 replies →
> What the feature Apple wanted to add was scan the files on the device and flag anything that gets a match.
This is not the revelation you think it is. Critics understood this perfectly.
People simply did not want their devices scanning their content against some opaque uninspectable government-controlled list that might send you to jail in the case of a match.
More generally, people usually want their devices working for their personal interests only, and not some opaque government purpose.
7 replies →
That technology of perceptional hashes could have failed in numerous ways, ruining lives of law-abiding users along the way.
7 replies →
Yes this is better than upload the entire photo. Just like virus scan can be done entirely on device, can flagging be local?. If homeomorphic encryption allows similarity matching, does not seem entirely private. Can people be matched?
> The intent was to protect data, children and reduce the amount of processing done on the server end to analyse everything.
If it’s for the children, then giving up our civil liberties is a small price to pay. I’d also like to give up liberties in the name of “terrorism”.
When we willingly give up our rights out of fear, these evil people have won.
1 reply →
> Everyone lost their mind yet it was clearly laid out in the papers Apple released on it.
And people working with CSAM and databases of CSAM have said it was a very bad idea.
4 replies →
> I don't understand most of the technical details of Apple's blog post.
I did understand the cited bits, and sorry to say but this could have been an optimistic post ("look at this cool new thing!")
I dislike Apple's anti-hacker (in the HN sense of the word) practices as much as the next person and don't own any Apple device for that and other reasons, but saying "it doesn't matter how you solved the privacy problem, I feel it's not private" doesn't make it true. Because most other people don't understand the cited words either, if they read that far down anyway, this seems like unfair criticism
Homomorphic encryption is something I heard about through a research paper a few years ago.
Back then I understood that an operation like SUM would be able to compute the sum of a list of numbers where each number was encrypted. The way the encryption worked made it possible to add all the values together without decrypting them, and the result ended up being encrypted too in such a way that the owner could decrypt it and have a number with a certain known accuracy.
If Apple is using homomorphic correctly then there should be no way for them to see the data they get from your phone. The other things they mention in the post as ways to prevent leaking of other information through metadata or a side channel.
The fact that this feature was enabled by default isn’t exactly great. Definitely should have been something that the user should have been asked if they wanted to enable after upgrading.
One specific use Apple is making of homomorphic encryption as of iOS 18 (I think) is for spam callers. You get a phone call, your phone sends Apple the encrypted phone number, they run it against their spam caller database, and you get the encrypted spam/not spam response back. They published a bunch of open source code around this functionality a while back.
https://www.swift.org/blog/announcing-swift-homomorphic-encr...
1 reply →
the main criticism is about sending private and sensitive data to Apple without consent and warning
I imagine Apple might argue that no private information is sent thanks to the use of homomorphic encryption. But Apple’s explanation rings hollow without the user having the ability to verify that this system is working as described.
2 replies →
How can you trust something you don't understand? That must come from "authority" (some person or org that you trust to know about such matters). That authority isn't Apple for many people. While I have cautious trust in Apple's privacy policies, many people don't, and not without reason. Hence, not understanding Apple's technical explanation of an Apple feature you didn't opt in to sharing personal data, increases the feeling of privacy violation (which in turn leads to more distrust).
So would it be unfair criticism?
> Hence, not understanding Apple's technical explanation of an Apple feature you didn't opt in to sharing personal data
But this is the fundamental issue. The author has no idea if personal data is being shared, they’ve made an assumption based on their lack of understanding. It’s entirely possible that all this service does (and arguably likely), is provide a private way for your phone to query a large database of landmark fingerprints, then locally try and match those fingerprints to your photos.
It doesn’t require send up private data. The phone could perform large geographic queries (the size of countries) for batches of fingerprints to be cached locally for photo matching. The homographic encryption just provides an added layer of privacy, allowing the phone to make those queries in a manner that makes it impossible for Apple to know what regions were queried for.
iOS photos already uses databases to convert a photo location into an address, so you can do basic location based searching. That will involve doing lookups in Apple global address database, do you consider that a violation of people’s privacy?
So you understand your own device’s security? You have no more reasons to trust the security of the Apple device in your pocket than you do of an Apple device in a datacenter IMHO.
> sorry to say but this could have been an optimistic post
> don't own any Apple device
So you don't have any skin in the game, but you're criticizing someone who does?
My blog post is written from the perspective of an Apple user whose trust has been violated. It's nice that you think—from a safe distance—the technology is neat, and maybe it is, but that's irrelevant to the main issue, which is the lack of user consent.
Hacker News unfortunately does not respond to this logic unless it is a company they are trained to hate. We could run the same story reporting Google and Meta's opt-out abuses, and it would also reach the frontpage with just as many comments. Except those comments would be violent condemnation, not apologetics and hand-wringing over whitepaper quotes.
It's tragic, because computing is in a professedly imperfect place right now. Digital privacy is under fire, many payments see a 30% digital service surcharge that is wholly arbitrary, and revolutionary cross-platform standards are being supplanted with proprietary and non-portable solutions that does not benefit any user.
As an American, I am ashamed that our government's dysfunction extends to consumer protection.
To me it seems like a reasonable feature that was, for the most part, implemented with great consideration for user privacy, though maybe I’m too trusting of the description. I mostly think this article is rage-bait and one should be wary of ‘falling for it’ when it shows up on hacker news in much the same way that one should be wary when rage-bait articles show up in tabloids or on Facebook.
It seems likely to me that concerns like those of the article or some of the comments in this thread are irrelevant to Apple’s bottom line. A concern some customers may actually have is data usage, but I guess it’s likely that the feature is off if in low data mode.
I wonder if this particular sort of issue would be solved by some setting for ‘privacy defaults’ or something where journalists/activists/some corporate IT departments/people who write articles like the OP can choose something to cause OS updates to set settings to values that talk less on the network. Seems hard to make a UI that is understandable. There is already a ‘lockdown mode’ for iOS. I don’t know if it affects this setting.
Literally all Apple needed to do was not have it enabled by default. Sending stuff over the network without asking is why trust in Apple is reduced further and further.
Not enabling something by default is pretty close to not having it at all. Accessibility is a reasonable exception where it makes sense to have the features even though they are off by default.
I mostly think the reaction to this article is overblown because it appeals popular ideas here about big tech. I think one should be wary of Apple’s claims about privacy: the reason is competition with Google and so they want users to be distrustful of the kinds of features that Google are better at implementing (I don’t want to say Apple isn’t trying to do the right thing either – if you look at accessibility, the competition was very bad for a lot of things for a long time and Apple was good despite the lack of commercial pressure). But I think one should also be wary of articles that make you angry and tell you what you suspected all along. (eg see the commenter elsewhere who doesn’t care about the details and is just angry). It’s much easier to spot this kind of rage-bait piece when it is targeting ‘normal people’ rather than the in-group.
7 replies →
Apple already communicates home by default. They never even fixed the macOS app signature check that they said they would, and yet people still choose to use the OS.
(And to be clear I’m not even bothered by the signature check)
At a certain point you have to figure that they realize it doesn’t matter short of some government entity forcing them to stop. At the very least the protections they put in place (homomorphic encryption, etc) are more than I think most other companies would ever bother doing.
[flagged]
5 replies →
It’s a reasonable feature, but should nevertheless require opt-in by the user. The opt-ins could certainly be bundled at install/upgrade time to reduce annoyance.
One thing particularly not clear to me is weather ios scan all data in the phone and send it to be part of public index or not. I see from how the feature works from the UI it seems it's not. If the feature activated by user action does this still constitute as phoning home?
if anyone else had done this then yes probably it's reasonable feature done reasonably. The problem is Apple has spent tens if not hundreds of millions of dollars advertising that they don't do things like this. That stuff stays on your iPhone unlike that other OS run by yucky advertising company. Apple would never siphon your data, because they care and you aren't the product.
Shit like this, reasonable in isolation or not, undermines that story completely. If they are so willing to just outright lie on a massive billboard, what else will they do when profits demand it?
I think I just noticed a similar thing for search that I'm pretty sure was not there before IOS 18.
Going into Settings -> Search there's an option now for "Help Apple Improve Search", enabled by default.
>Help improve Search by allowing Apple to store the searches you enter into Safari(!!), Siri and Spotlight in a way that is not linked to you. Searches include lookups of general knowledge, and requests to do things like play music and get directions.
If it was there before then it was switched on again.
> allowing Apple to store the searches you enter into Safari […] in a way that is not linked to you
From deanonymization work even a decade ago it was clear that your search history will completely unmask you.
I would need lots of details before I believed that their method of storing the data reliably protected my privacy. (But of course that is not what the quote claims.)
This is worse than the OP. What alternative universe is Apple living in where it thinks that defaulting to collecting people’s search histories without consent is OK?
Oh damn you’re right. This was turned on by default for me, I don’t think I would have opted into this.
You’re right. This is another setting turned on by default.
thanks man
As trivia, on mac os, the photoanalysisd service will run in the background and look through your photos, even if you never open Apple Photos. It can't be disabled unless you disable SIP (system integrity protection) which requires a complicated dance of reboots and warnings. It will reenable if you turn SIP back on.
It seems Apple are very passionate about analysing your photos for some reason, regardless if you yourself are.
You may also be shocked to learn that Spotlight looks through every single file on your Mac.
I was. First by the md_worker processes that mysteriously started pinning all of my CPU cores after a git clone. Then by the realization that MacOS had built a full-text index of millions of lines of source code (it only took a few hours of my Mac being too hot to touch).
A lot of Apple's defaults are just plain bizarre. Why the hell is Spotlight seeing source code mimetypes and feeding it to the search index?
6 replies →
Local search indexing is somewhat more defendable as a system level service, but yeah, it would be nice if that was also up to me as a user.
Isn’t that running locally on your Mac though?
CSAM could already be part of some local service theoretically. Privacy ended with a requirement to have an account linked to the device (not just icloud). There is no account needed to use a Linux computer.
You don't need to use icloud to use a Mac.
Does it phone home? I don't care about scanning my files. I do care about details of my private data leaving my device.
“It seems Apple are very passionate about analysing your photos for some reason, regardless if you yourself are.”
Isn’t this fragile to pollution from end users?
What if we all ran A local Image generator trained on our own photos… But slightly broken… And just flooded their photo hash collection with garbage?
Now what ?
This would be a very good flushing action. Lot would be learned by seeing who got angry about this and how angry they got…
No. The analysis in question is fully local, used for indexing photos by categories in the Photos app. It is unrelated to any cloud features and not something shared across users.
They are also not using your personal photos to feed the location database, most likely public sources and/or Apple Maps data. If they are relying on GPS-tagged public photos alone, you could probably mess up a system like this by spoofing GPS location en-masse and posting them online for years, but for what purpose?
1 reply →
> for some reason
It's directly tied to features they provide in their photo app. This is hardly obscure.
Why make it extremely hard to disable? Photos is hardly a system level app
1 reply →
All kinds of nonsense runs and phones home throughout the os. The thing that annoyed me the most is trying to create an account will phone home to apple, such as setting up a local smtp/imap server on the local network.
This whole thing is reminding me of the outrage over Apple and Google's privacy preserving 'Exposure Notification System' system from the Covid years. It defies intuition that they can alert you to exposure without also tracking you, but indeed that's what the technology lets you do.
Similarly here, it feels like the author is leaning into a knee jerk reaction about invasion of privacy without really trying to evaluate the effectiveness of the technologies here (client side vectorization, differential privacy, OHTTP relays, and homomorphic encryption).
Though I 100% agree Apple should ask the user for consent first for a feature like this.
I would love to evaluate the privacy of these technologies.
Someone reply with a link to the source code so I can see exactly what it is doing, without having to take an internet rando's word for it.
Better yet, let me compile it myself.
You can start with this https://github.com/apple/swift-homomorphic-encryption
Of course it is not the whole technology stack, but it is something at least. If your evaluation leads to potential problems, you can create issues right there on the github project!
If you have the capability to actually skillfully analyze this type of crypto, disassembling the binaries from your device (or at the very least, an ipsw for your device) should be trivial.
After all, you wouldn’t actually be trusting the source code given to you to match what’s running on your device, would you?
1 reply →
You had better build your own silicon chips and phone hardware as well in that case.
6 replies →
That COVID feature was opt-in. Author is complaining about a lack of opt in now.
"I don't understand most of the technical details of Apple's blog post"
I do:
- Client side vectorization: the photo is processed locally, preparing a non-reversible vector representation before sending (think semantic hash).
- Differential privacy: a decent amount of noise is added the the vector before sending it. Enough to make it impossible to reverse lookup the vector. The noise level here is ε = 0.8, which is quite good privacy.
- OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
- Homomorphic encryption: The lookup work is performed on server with encrypted data. Apple can't decrypt the vector contents, or response contents. Only the client can decrypt the result of the lookup.
This is what a good privacy story looks like. Multiple levels of privacy security, when any one of the latter 3 should be enough alone to protect privacy.
"It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here.
I'm not sure what the right call is here. But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true. There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk. But if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
I appreciate the explanation. However, I think you do not address the main problem, which is that my data is being sent off my device by default and without any (reasonable) notice. Many users may agree to such a feature (as you say, it may be very secure), but to assume that everyone ought to be opted in by default is the issue.
I'm not sure I agree -- asking users about every single minor feature is (a) incredibly annoying, and (b) quickly causes request-blindness in even reasonably security-conscious users. So restraining the nagging for only risky or particularly invasive things makes sense to me.
Maybe they should lump its default state into something that already exists? E.g. assume that if you already have location access enabled for Photos (it does ask!), you've already indicated that you're okay with something about this identifying being sent to Apple whenever you take a picture.
My understanding is that Location Services will, among other things, send a hash of local WiFi network SSIDs and signal strengths to a database Apple maintains, and use that to triangulate a possible position for you. This seems loosely analogous to what's going on here with the compute-a-vector thing.
22 replies →
I think it does address the main problem. What he is saying is that multiple layers of security is used to ensure (mathematically and theoretically proved) that there is no risk in sending the data, because it is encrypted and sent is such a way that apple or any third party will never be able to read/access it (again, based on theoretically provable math) . If there is no risk there is no harm, and then there is a different need for ‘by default’, opt in/out, notifications etc.
The problem with this feature is that we cannot verify that Apple’s implementation of the math is correct and without security flaws. Everyone knows there is security flaws in all software, and this implementation is not open (I.e. we cannot review the code, and even if we could review code we cannot verify that the provided code was the code used in the iOS build). So, we have to trust Apple did not make any mistakes in their implementation.
35 replies →
I’m a cryptographer and I just learned about this feature today while I’m on a holiday vacation with my family. I would have loved the chance to read about the architecture, think hard about how much leakage there is in this scheme, but I only learned about it in time to see that it had already been activated on my device. Coincidentally on a vacation where I’ve just taken about 400 photos of recognizable locations.
This is not how you launch a privacy-preserving product if your intentions are good, this is how you slip something under the radar while everyone is distracted.
8 replies →
I think I'm saying: you're not sending "your data" off device. You are sending a homomorphically encrypted locally differentially private vector (through an anonymous proxy). No consumer can really understand what that means, what the risks are, and how it would compare to the risk of sending someone like Facebook/Google raw data.
I'm asking: what does an opt in for that really look like? You're not going to be able to give the user enough info to make an educated decision. There's ton of risk of "privacy washing" ("we use DP" but at very poor epsilon, or "we use E2E encryption" with side channel data gathering).
There's no easy answer. "ask the user", when the question requires a phd level understanding of stats to evaluate the risk isn't a great answer. But I don't have another one.
53 replies →
Notice is always good and Apple should implement notice.
However, "my data is being sent off my device" is incorrect, as GP explained. Metadata, derived from your data, with noise added to make it irreversible, is being sent off your device. It's the equivalent of sending an MD5 of your password somewhere; you may still object, but it is not factually correct to say your password was transmitted.
16 replies →
"Your data" is not actually being sent off your device, actually, it is being scrambled into completely unusable form for anyone except you.
This is a much greater level of security than what you would expect from a bank, for example, who needs to fully decrypt the data you send it. When using your banking apps over HTTPS (TLS), you are trusting the CA infrastructure, you are trusting all sorts of things. You have fewer points of failure when a key for homomorphic encryption resides only on your device.
"Opting-in by default" is therefore not unsafe.
I guess it depends on what you're calling "your data" -- without being able to reconstruct an image from a noised vector, can we say that that vector in any way represents "your data"? The way the process works, Apple makes their own data that leaves your device, but the photo never does.
2 replies →
How would you explain client side vectorization, differential privacy and homomorphic encryption to a layman in a single privacy popup so that they can make an informed choice?
Or is it better to just trust that mathematics works and thus encryption is a viable way to preserve privacy and skip the dialog?
The big mistake here is ownership of your apple devices is an illusion...
Do you consider your data to include non-reversible hashes of your data injected with random noise? I'm not sure I consider that my data. Its also not even really meta-data about my data.
Do you use iCloud to store your photos?
50 replies →
When your phone sends out a ping to search for cellular towers, real estate brokers collect all that information to track everywhere you go and which stores you visit.
Owning a phone is a privacy failure by default in the United States.
3 replies →
> This is what a good privacy story looks like.
What a good privacy story looks like is that my photos aren’t sent anywhere in any way shape or form without explicit opt in permission.
Your photos aren't sent anywhere in this system.
3 replies →
Do you not sync to iCloud?
7 replies →
You're presenting a false dichotomy between "perfect user understanding" and "no user choice." The issue isn't whether users can fully comprehend homomorphic encryption or differential privacy – it's about basic consent and transparency.
Consider these points:
1. Users don't need a PhD to understand "This feature will send data about your photos to Apple's servers to enable better search."
2. The complexity of the privacy protections doesn't justify removing user choice. By that logic, we should never ask users about any technical feature.
3. Many privacy-conscious users follow a simple principle: they want control over what leaves their device, regardless of how it's protected.
The "it's too complex to explain" argument could justify any privacy-invasive default. Would you apply the same logic to, say, enabling location services by default because explaining GPS technology is too complex?
The real solution is simple: explain the feature in plain language, highlight the benefits, outline the privacy protections, and let users make their own choice. Apple already does this for many other features. "Default off with opt-in" is a core principle of privacy-respecting design, regardless of how robust the underlying protections are.
I don't believe I said or implied that anywhere: 'You're presenting a false dichotomy between "perfect user understanding" and "no user choice."'? Happy to be corrected if wrong.
Closest I come to presenting an opinion on the right way UX was "I'm not sure what the right call is here.". The thing I disagreed with was a technical statement "the only way to guarantee computing privacy is to not send data off the device.".
Privacy respecting design and tech is a passion of mine. I'm pointing out "user choice" gets hard as the techniques used for privacy exceed the understanding of users. Users can intuitively understand "send my location to Google [once/always]" without understanding GPS satellites. User's can't understand the difference between "send my photo" and "send homomorphicly encrypted locally differentially private vector of e=0.8" and "send differentially private vector of e=50". Your prompt "send data about your photos..." would allow for much less private designs than this. If we want to move beyond "ask the user then do it", we need to get into the nitty gritty details here. I'd love to see more tech like this in consumer products, where it's private when used, even when opted-in.
3 replies →
> This is what a good privacy story looks like.
A good privacy story actually looks like not sending any info to anyone else anywhere at any time.
Your answer shows how we all have a very different idea of what our own desired privacy level is. Or what privacy even means.
2 replies →
Sure, but if we follow that line of thinking to its logical conclusion, we must move to a cabin in the woods, 100 miles from the nearest civilization, growing our own food and never connecting our computing devices to anything resembling a network.
18 replies →
> The author themselves looks to be an Apple security researcher
They’re not. Jeff Johnson develops apps (specifically Safari extensions) for Apple platforms and frequently blogs about their annoyances with Apple, but they’re not a security researcher.
Thank you for this comment. I found the author's ignorance to be fairly discrediting, and was surprised to find so many follow up comments equally railing on Apple.
Between the quote you pointed out and:
"One thing I do know, however, is that Apple computers are constantly full of privacy and security vulnerabilities, as proved by Apple's own security release notes" which just reeks of survivorship bias.
I think the final call of what is right here _shouldn't_ be informed by the linked article.
IMO, enabled by default without opt-in is absolutely the right call when judging between 1: Feature value 2: Security risk 3: Consent Fatigue.
If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
If you disagree with my prior statement because of how Apple locks you into Photos, :shake_hands:.
If Enhanced Visual Search is still enabled by default in lockdown mode, then I think that's worth a conversation.
> I found the author's ignorance to be fairly discrediting
Why in the world am I supposed to be an expert on homomorphic encryption? How many people in the world are experts on homomorphic encryption?
> which just reeks of survivorship bias.
What does that even mean in this context?
> 1: Feature value
What is the value of the feature? As the article notes, this new feature is flying so low under the radar that Apple hasn't bothered to advertise it, and the Apple media haven't bothered to mention it either. You have to wonder how many people even wanted it.
> If you're data-conscious enough to disagree with my prior statement, you should consider having lockdown mode enabled.
That's ridiculous. Apple itself has said, "Lockdown Mode is an optional, extreme protection that’s designed for the very few individuals who, because of who they are or what they do, might be personally targeted by some of the most sophisticated digital threats. Most people are never targeted by attacks of this nature." https://support.apple.com/105120
Lockdown mode is basically for famous people and nobody else.
4 replies →
Enhanced Visual Search was enabled despite my default lockdown mode. I worry about enhanced visual search capabilities much less than several of the other risky features that lockdown mode disables, but was a bit surprised by the default opt-in in my lockdown mode phone.
1 reply →
> This is what a good privacy story looks like.
A good privacy story starts with "Do you consent" and not transmitting a byte if you answer "no"
This sounds exactly like that CSAM "feature" they wanted to add but created a huge outrage because of how incredibly invasive it was.
It sounds like it only needs a few extra lines of code to get exactly what they wanted before, they just packaged it differently and we all fell for it like frogs getting boiled in water.
A frog that is gradually heated will actually jump out.
https://en.m.wikipedia.org/wiki/Boiling_frog
1 reply →
Oh ffs, not this one again.
The CSAM filtering was a best of class implementation. I'm pretty sure I'm one of maybe a dozen people who actually read the spec before throwing a hissy-fit about "muh privacy!"
The only actual "flaw" was that maybe a state-level actor could make it scan for bad stuff on your device.
BUT they can do it to your cloud data _today_. And if you disabled cloud uploads, the local scanning was disabled too.
1 reply →
I’m deeply familiar with all of these techniques, the core issue here is informed consent which they have not obtained.
Furthermore, Apples privacy stance is generally a sham as their definition of “human rights” doesn’t extend to China. Which either means Apple doesn’t respect human rights, or they don’t view Chinese people as human.
Apple follows the law. First you need to get the Chinese government to respect those rights. The only other choice is to stop doing business entirely in the country.
1 reply →
That's not really fair; Apple's in a sticky wicket when it comes to the Chinese government, and they're not the only ones.
The Chinese government are debatably inhuman. They've literally censored the word "censorship." (Then they censored what people used euphemistically for censorship--"harmonious.") It's funny from the outside but also a miserable state of affairs in 2024.
3 replies →
The nearest neighbour search is sharded, which apple's blog admits is a privacy issue, which is why they're running the DP and OHTTP parts.
If apple were to add additional clusters that match "sensitive" content and endeavour to put them in their own shards distinct from landmarks, they defeat the homomorphic encryption part while still technically doing it.
The DP part can be defeated with just statistics over time; someone with any volume of sensitive content will hit these sensitive clusters with a higher likelihood than someone generateing noise injected fake searches.
The OHTTP part can be defeated in several ways, the simplest of which is just having a clause in a non-public contract allowing apple to request logs for some purpose. They're paying them and they can make up the rules as they go.
This must be the first consumer or commercial product implementing homomorphic encryption is it not?
I would be surprised if doing noisy vector comparisons is actually the most effective way to tell if someone is in front of the Eiffel tower. A small large language model could caption it just as well on device, my spider sense tells me someone saw an opportunity to apply bleeding edge, very cool tech so that they can gain experience and do it bigger and better in the future, but they're fumbling their reputation by doing this kind of user data scanning.
> This must be the first consumer or commercial product implementing homomorphic encryption is it not?
Not really, it's been around for a bit now. From 2021:
> The other major reason we’re talking about HE and FL now is who is using them. According to a recent repository of PETs, there are 19 publicly announced pilots, products, and proofs of concept for homomorphic encryption and federated analytics (another term for federated learning) combined. That doesn’t seem like a lot … but the companies offering them include Apple,7 Google, Microsoft, Nvidia, IBM, and the National Health Service in the United Kingdom, and users and investors include DARPA, Intel, Oracle, Mastercard, and Scotiabank. Also, the industries involved in these early projects are among the largest. Use cases are led by health and social care and finance, with their use in digital and crime and justice also nontrivial (figure 1).
https://www2.deloitte.com/us/en/insights/industry/technology...
I do wonder why we don't hear about it more often though. "Homomorphic encryption" as a buzzword has a lot of headline potential, so I'm surprised companies don't brag about it more.
2 replies →
Apple themselves have already used it in the past (Caller ID)
It seems apple might be using it for live caller id lookup?
> This is what a good privacy story looks like.
I have an idea: send an encrypted, relayed, non-reversible, noised vector representation of your daily phone habits and interactions. That way you can be bucketed, completely anonymously of course, with other user cohorts for tracking, advertising, and other yet-to-be discovered purposes.
It's a great privacy story! Why would you have a problem with that?
What would be the value to the user in your scenario? In the photos app real scenario, it’s to enable a search feature that requires pairing photos with data not on the phone. (I understand you’re being sarcastic.)
4 replies →
They don't "have your data," even at an aggregated and noised level, due to the homomorphic encryption part.
Restating the layers above, in reverse:
- They don't see either your data or the results of the query (it's fully encrypted even from them where they compute the query -- this is what homomorphic encryption means)
- Even if they broke the encryption and had your query data / the query result, they don't know who "you" are (the relay part)
- Even if they had your query hash and your identity, they couldn't reverse the hash to identify which specific photos you have in your library (the client-side vectorization + differential privacy part), though by the this point they could know what records in the places database were hits. So they could know that you took a photo of a landmark, but only if the encryption and relay were both broken.
I am bit bit confused: Data is being sent to Apple, in such a way that it can not be traced back to the user. Apple does some processing on it. Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....
You joked, but you accidentally described what homomorphic encryption does. (if implemented correctly)
> Then somehow magically, the pictures on your phone are updated with tags based on Apple's processing....but Apple doesn't know who you are.....
Yes, this is the whole point.
There is a way to perform processing on encrypted data so the result is also encrypted and the person doing the processing never knows anything about the data that was processed on or the result (which can only be decrypted by the user with the original encryption keys)
https://en.wikipedia.org/wiki/Homomorphic_encryption
> Homomorphic encryption is a form of encryption that allows computations to be performed on encrypted data without first having to decrypt it. The resulting computations are left in an encrypted form which, when decrypted, result in an output that is identical to that produced had the operations been performed on the unencrypted data. Homomorphic encryption can be used for privacy-preserving outsourced storage and computation. This allows data to be encrypted and outsourced to commercial cloud environments for processing, all while encrypted
And the way the data comes back to you is via the third-party relay which knows your IP but nothing else
1 reply →
For context, @scosman is self-described as “Formerly Apple Photos” in his Twitter bio.
The devil is in the differential privacy budget. I am in Japan and I’ve taken hundreds of photos this week. What does that budget cover?
> - OHTTP relay: it's sent through a 3rd party so Apple never knows your IP address. The contents are encrypted so the 3rd party never doesn't learn anything either (some risk of exposing "IP X is an apple photos user", but nothing about the content of the library).
Which 3rd party is that?
I don't have a list on hand, but at least Cloudflare and Akamai are part of the network hops. Technically you only need 2 hops to make sure no origin or data extraction can be done.
3 replies →
The NSA, the CCP, etc. depending on jurisdiction. (joking, but not really)
>There are other tools to provide privacy (DP, homomorphic encryption), while also using services. They are immensely complicated, and user's can't realistically evaluate risk.
It is simple for any user to evaluate risk the risk of their data being breached on 3rd party servers when their data isn't being sent off the device - there is none. It is only when corporations insist that they are going to send the data off your device whether you like it or not that evaluating risk becomes necessary.
Regarding HE: since the lookup is generated by the requestor, it can be used as an adversarial vector, which can result in exfiltration by nearest neighbor (closest point to vector) methods. In other words, you can change what you are searching for, and much like differential power analysis attacks on crypto, extract information.
Does the noise addition not help? Is that a mitigation for a different attack?
Isn’t this 95% the same as Apple’s CSAM scanning paper?
It’s a nice story but Apple can change the implementation any time they want or are forced to.
This answer should be much higher. Thank you
This may be a "good" privacy story but a way better one is to just not send any of your data anywhere, especially without prior consent.
The best you can hope is integrity and security until your information reaches the destination but to assume that Apple or the U.S government cannot decipher the information you sent it or use it against you(i.e. set a person of interest as "landmark" and find out who's iPhone matches that "landmark) you must be foolish.
It's no longer a conspiracy. I think we are all over past that time(i.e with Snowden and Wikileaks). We live in a surveillance world and "They're guarding all the doors and holding all the keys".
> This is what a good privacy story looks like.
Not at all. A good privacy story is not sending this data anywhere.
> I'm not sure what the right call is here.
I am sure.
The right call is to never send any data from the device to anyone unless the user explicitly tells the device to do it.
The only thing the device should do is whatever its user tells it to do.
The user didn't tell it to do this. Apple did.
> But the conclusion "Thus, the only way to guarantee computing privacy is to not send data off the device." isn't true
Irrelevant. It was never about privacy to begin with. It was always about power, who owns the keys to the machine, who commands it.
Vectorization, differential privacy, relays, homomorphic encryption, none of it matters. What matters is the device is going behind the user's back, doing somebody else's bidding, protecting somebody else's interests. That they were careful about it offers little comfort to users who are now aware of the fact "their" devices are doing things they weren't supposed to be doing.
Complete nonsense. *All networked devices do things behind their users back* at this point, and have for years, and do not ask for consent for most of it. And users would REJECT granular opt-in as a terrible UX.
Let's look at the primary alternative, Android. It generally does not provide you this level of granular control on network access either without rooted hacks. Apps and the phone vendor can do whatever they want with far less user control unless you're a deep Android nerd and know how to install root-level restriction software.
1 reply →
> if you want features that require larger-than-disk datasets, or frequently changing content, you need tools like this.
Well I want them to fuck off.
Hidden in your commentary here is the fact that the vector representation of the image is _the contents of the image_. It very well may be that they cannot reverse the exact image. But it’s still a representation of the image that has to be good for something. Without being too familiar I would be willing to hazard a guess that this could include textual labels and classifications of what is in the image.
I don’t give a shit how good your internal controls are. I don’t have anything particularly interesting to hide. I still do not want you taking my pictures.
If you read the research you'd know that they don't have access to the vector either. They never decrypt the data. All operations on their server are done directly on the encrypted data. They get 0 information about your photos. They cannot even see which landmark your vector was closest to.
2 replies →
> "It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations." -> The author themselves looks to be an Apple security researcher, and are saying they can't make an informed choice here
I don’t think that that’s what the author is saying at all, I think he’s saying that Apple should let the user decide for themself if they want to send all this shit to Apple, freedom for the individual. They’re not saying “I dunno”
Can this be verified?
this is what gaslighting looks like
how about they don't send anything about my photos to their servers and i get to keep my shit on my own device
i suppose we're past that to the point where techbros like you will defend personal data exfiltration because.. uhh idk? trillion dollar corporation knows best?
So what? Why should the application talk over the Internet to begin with? And why isn't that functionality off by default under a settings option that clearly warns the user of the consequences? I think you're missing the forest for the trees here.
And the claims that this is good privacy/security are not at all obvious either. And who are those third-parties anyway? Did you verify each one of them?
Quantum makes the homomorphic stuff ineffective in the mid-term. All they have to do is hold on to the data and they can get the results of the lookup table computation, in maybe 10-25 years. Shouldn't be on by default.
What makes you think that this is the biggest problem if things like AES and RSA are suddenly breakable?
If someone wanted to get a hold of your cloud hosted data at that point, they would use their capacity to simply extract enough key material to impersonate a Secure Enclave. That that point, you "are" the device and as such you "are" the user. No need to make it more complicated than that.
In theory, Apple and other manufacturers would already use PQC to prevent such scenarios. Then again, QC has been "coming soon" for so long, it's doubtful that any information that is currently protected by encryption will still be valuable by the time it can be cracked. Most real-world process implementations don't rely on some "infinite insurance", but assume it will be breached at some point and just try to make it difficult or costly enough to run out the clock on confidentiality, which is all that really matters. Nothing that exists really needs to be confidential forever. Things either get lost/destroyed or become irrelevant.
1 reply →
The right call is to provide the feature and let users opt-in. Apple knows this is bad, they've directly witnessed the backlash to OCSP, lawful intercept and client-side-scanning. There is no world in which they did not realize the problem and decided to enable it by default anyways knowing full-well that users aren't comfortable with this.
People won't trust homomorphic encryption, entropy seeding or relaying when none of it is transparent and all of it is enabled in an OTA update.
> This is what a good privacy story looks like.
This is what a coverup looks like. Good privacy stories never force third-party services on a user, period. When you see that many puppets on stage in one security theater, it's only natural for things to feel off.
> This is what a coverup looks like.
That’s starting to veer into unreasonable levels of conspiracy theory. There’s nothing to “cover up”, the feature has an off switch right in the Settings and a public document explaining how it works. It should not be on by default but that’s not a reason to immediately assume bad faith. Even the author of the article is concerned more about bugs than intentions.
11 replies →
> This is what a coverup looks like
This is a dumb take. They literally own the entire stack Photos runs on.
If they really wanted to do a coverup we would never know about it.
4 replies →
It's not that binary. Nobody is forcing anything, you can not buy a phone, you can not use the internet. Heck, you can even not install any updates!
What is happening, is that people make tradeoffs, and decide to what degree they trust who and what they interact with. Plenty of people might just 'go with the flow', but putting what Apple did here in the same bucket as what for example Microsoft or Google does is a gross misrepresentation. Present it all as equals just kills the discussion, and doesn't inform anyone to a better degree.
When you want to take part in an interconnected network, you cannot do that on your own, and you will have to trust other parties to some degree. This includes things that might 'feel' like you can judge them (like your browser used to access HN right here), but you actually can't unless you understand the entire codebase of your OS and Browser, all the firmware on the I/O paths, and the silicon it all runs on. So you make a choice, which as you are reading this, is apparently that you trust this entire chain enough to take part in it.
It would be reasonable to make this optional (as in, opt-in), but the problem is that you end up asking a user for a ton of "do you want this" questions, almost every upgrade and install cycle, which is not what they want (we have had this since Mavericks and Vista, people were not happy). So if you can engineer a feature to be as privacy-centric yet automated as possible, it's a win for everyone.
15 replies →
I don't care if the device checks it, it uses homomorphic encryption and differential privacy. This is turned on by default. This angers me.
Clearly, in Cupertino, 'enhancing user experiences' without consent is the top priority.
"homomorphic encryption and differential privacy"
It is new. It is fancy. It is not clear where HE/DP is being used, it depends if the code is written using the Swift toolkit, but even that has paths for exfiltration if used incorrectly. They claim they are using DP in Photos as stated in the article here:
https://machinelearning.apple.com/research/scenes-differenti...
But the fact remains they are looking at your pictures. I do not trust them for one fleeting fart in the wind on this. Think about it for a hot second: HE/DP allows you to perform operations on the data without knowing the data, but what if someone goofs an operation and it ends up returning the actual data?
Sorry, not buying it. Crypto is hard to get right, and when it is monetized like this for "new features", it is wildly unnecessary and exposes users to more risk.
> what if someone goofs an operation and it ends up returning the actual data
That's kind of like saying "they can take a picture of your face and transmit it, but what if someone goofs an operation and sends your actual head instead".
Any encrypted data the server has cannot 'accidentally' be decrypted, and as someone else explained in this thread they only send encrypted vectors describing features of the image (building, tree, etc) and not the actual image. It's certainly not a fact that "they are looking at your pictures" [1].
[1] "they" being Apple; the other guys could have backdoored the whole OS and Apple employee's computers for all I know
So basically - You take a picture. Apple encrypts it and uploads it to their server. The server matches the (still encrypted) picture to a database and tells your device "this picture contains the Eiffel Tower". Later when you search for Eiffel Tower on your device the photo pops up.
Is the complexity and security risk really worth it for such a niche feature?
It's also funny that Apple is simultaneously saying "don't worry the photo is encrypted so we can't read it" and "we are extracting data from the encrypted photo to enhance your experience".
They don’t send the photo. They send some encrypted metadata to which some noise is added. The metadata can be loosely understood as “I have this photo that looks sort of like this”. Then the server takes that encrypted data from the anonymized device and responds something like “that looks like the Eiffel Tower” and sends it back to the device. The actual photo never goes to the server.
With the added caveat that HE is magic sauce - so the server cannot see the metadata (cropped/normalized image data), and doesn't know how much it does or does not look like the Eiffel Tower.
They don’t send the photos. Nobody sent your photos anywhere but only certain meta data and its similarity vectors for matching purpose.
How cannot tell the picture contains the Eiffel Tower if the image is not decrypted?
Because it turns out that mathematicians and computer scientists have devised schemes that allow for certain computational operations to be performed on encrypted data without revealing the data itself. You can do a+b=c and it doesn’t reveal anything about what a and b are is the intuition here. This has been mostly confined to the realm of theory and mathematics until very recently but Apple has operationalized it for the first time.
6 replies →
Is this a niche feature? I use this kind of search very often in my photos.
What are some example keywords? I have never searched for landmarks, I only search for location.
How many landmarks are there to justify sending your data off? Can't the database be stored on the device?
3 replies →
Not really. It's more like apple runs a local algorithm that takes your picture of the Eiffel tower, and outputs some text "Eiffel tower, person smiling", and then encrypts that text and sends it securely to apples servers to help you when you perform a search.
OP was wrong, but this is even wronger
Locally, a small ML model identifies potential POIs in an image.
Another model turns these regions into a series of numbers (a vector) that represent the image. For instance, one number might correlate with how "skyscraper-like" the image is. (We don't actually know the definition of each dimension of the vector, but we can turn an image that we know is the eiffel tower into a vector, and measure how closely our reference image and our sample image are located)
The thing is, we aren't storing this database with the vectors of all known locations on our phone. We could send the vector we made on device off to Apple's servers. The vector is lossy, after all, so apple wouldn't have the image. If we did this, however, apple would know that we have an image of the eiffel tower.
So, this is the magic part. The device encrypts the vector using a private key known only to it, then sends this unreadable vector off to the server. Somehow, using Homomorphic Encryption and other processes I do not understand, mathematical operations like cosine similarity can be applied to this encrypted vector without reading the actual contents of the vector. Each one of these operations changes the value, which is still encrypted, but we do not know how the value changed.
I don't know if this is exactly what Apple does, I think they have more efficient ways, but theoretically what you could do is apply each row in your database to this encrypted value, in such a way that the encrypted value becomes the name of the POI of the best match, or otherwise junk is appended (completely changing the encrypted value) Again, the server has not read the encrypted value, it does not know which row won out. Only the client will know when it decrypts the new value.
This discussion got quite long without anyone mentioning the novel technical implementation paper "Scalable Private Search with Wally". Kudos to Apple for researching this, but shame on them for enabling this by default.
As a somewhat homomorphic encryption scheme, each query releases some bits of information on what you searching for to avoid using the whole database. Subsequent queries from a given user will generally be correlated enough to build a tracer. Governments or other powerful enough actors can pierce the proxy veil, heck the tracer will be able to deanonymize you with enough queries recorded.
How many queries? For me it is too tedious to work out the math from the differential privacy definitions and I already know the landmarks around me: I don't want such a feature.
Hi,
Very curious here as I haven’t seen any papers demonstrating attacks against the differential privacy systems proposed by Apple or Google that successfully deanonymize data. Such an attack in even a test database would be very interesting.
Do you have any papers you can cite about this entropic leakage you’re describing?
The very difference between somewhat and full homomorphic encryption hinges on this leakage as explained in the paper. The definition of differential privacy as well. They directly admit to leaking a certain amount of information by stating that they apply differential privacy with those given parameters. The issue I am talking about is that such concerns are applied on a single query but correlations across query (the things that actually happen with metadata) aren't considered in the delta-epsilon differential privacy model, by definition.
So if you are curious, just read up in general about differential privacy, or this classic: https://gwern.net/death-note-anonymity Discussed here (possibly in other places too): https://news.ycombinator.com/item?id=9553494
I don’t even use iCloud Photos and this was on by default. Very bad move by Apple to ship my photos off my device, without my permission, in any shape or form, I don’t care.
If you don't use iCloud Photos your photos are not shipped off your device.
I wish it were so! That was true until they turned this feature on by default.
That isn't what is happening. The author of this blog post has absolutely no idea what he is talking about.
> That isn't what is happening
Then why is it happening to me?
It’s funny how the wordsmiths come out to defend Apple here.
Your “photos” aren’t shipped off-device without your knowledge, just an arbitrary blob of ”metadata” that you can’t audit describing everything about that photo. :)
It’s sort of like “I don’t want my WiFi router uploading my house online!” And someone replying “it’s not your physical house, just a map of the house and the real time location of everyone in it! The house never moves!”
Another setting that surprised me with being turned on by default apparently on macOS 15 is System Settings - Spotlight - "Help Apple Improve Search": "Help improve Search by allowing Apple to store your Safari, Siri, Spotlight, Lookup, and #images search queries. The information collected is stored in a way that does not identify you and is used to improve search results."
No, this is not on by default. After system install at first boot it asks if you want to help improve search and it describes how your data will be handled, anonymized, etc. If you clicked on yes it is on. There is a choice to opt out.
I was on by default for me, both after the macOS 14 -> 15 upgrade and after installing macOS 15 cleanly. I wonder if they ask for consent in some regions only.
1 reply →
You're lying, it is on by default.
I always say no to all analytics/improvement prompts.
Both the search and photos settings were on for me in both iOS and MacOs. Completely unacceptable Apple
I love the, "help <megacorp> do x" options. Help? Your market cap is over $3tn. Help yourself! And stop asking how my call quality was.
At this point, Mac Mini M4's are cheap enough and capable enough to just purchase two: one for off-line use, another on-.
Perhaps this is marketing genius (from an AAPL-shareholder POV)?
----
I'm laughing at the insanity of all this interconnectivity, but an NDA prevents me from typing the greatest source of my ironic chuckles. Described in an obtuse way: a privacy-focused hardware product ships with an undisclosed phone-home feature, letting the feds see every time you use the product (to produce a controversial product, at home).
Kick in my fucking door / sue me: it'll just re-enforce that I'm correct about concessionary-allowances...
Can you be more clear?
Computer are inexpensive enough to own both on- & off-line hardware.
----
Even privacy-focused hardware manufacturers will allow undisclosed usage tracking (in order to continue existing, themselves). In my example, the OEM delivers a physical product which allows users to make tangible objects, at home. Every time you launch the hardware's control software (to make more controversial objects), it phones home.
> letting the feds see every time you use the product
This does not happen.
Traffic analysis by the government totally happens: https://lifehacker.com/how-cia-director-david-petraeuss-emai...
Now they probably don't care about you personally, but you'd be surprised how many people are connected indirectly to a person of interest. Google "Maher Arar" for the horrific consequences of false positives.
1 reply →
Wrong. Anything that makes any network request is by definition a privacy leak. The network itself is always listening, and the act of making any connection says that you are using the computer, and where, and when.
10 replies →
[dead]
> On macOS, I can usually prevent Apple software from phoning home by using Little Snitch. Unfortunately, Apple doesn't allow anything like Little Snitch on iOS.
On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
Apple does publish IP ranges for different services, so it's theoretically possible to block 17.0.0.0/8 and then open up connections just for notifications and security updates, https://support.apple.com/en-us/101555
An iOS "local VPN" could definitely block all traffic to Apple IP ranges. But it lacks the ability to associate traffic with the originating process/framework. Like if, for example, I wanted to only allow iMessage to talk to Apple but nothing else. This is what Little Snitch and other software gives you on macOS/Linux/etc.
But even blanket blocking of all Apple IP ranges probably wouldn't do anything here. As documented, your device sends noise injected image vectors to OHTTP relays and doesn't contact Apple directly. By definition those relays are operated by 3rd parties. So if you consider this type of data "phoning home" you'll need to find the IPs of all of OHTTP relays iOS uses. (or block the traffic that looks up the OHTTP relays).
Apple's "enterprise networking" guide lists 3rd-party CDNs as subdomains of apple.com, which usually resolve to akamai or cloudflare subdomains. This allows those dynamic IPs to be blocked via dnsmasq ipset rules. In theory, they could use similar subdomain resolution for the OHTTP relays.
Since iOS was derived from macOS, perhaps Apple could restore the link between network traffic and process.
It looks sensible assuming that Little Snitch has some high level manager agent inside Apple manipulating the company making these kind of sneaky attacks on customers' privacy that drives the sales of Little Snitch. On the end they will also make them to buy Liitle Snitch for lots of millions or billions for elimination so they can attack customers freely afterwards. Little Snitch hidden agents are smart!
I do not assume that Apple managers are that degenerate idiots pushing through trust eroding marginal idiocy like this.
Apple probably wouldn't allow such an app in their walled garden?
>On Android, NetGuard uses a "local VPN" to firewall outgoing traffic. Could the same be done on iOS, or does Apple network traffic bypass VPNs? Lockdown mentions ads, but not Apple servers, https://lockdownprivacy.com/.
Why is NetGuard more trustworthy than Apple?
NetGuard firewall doesn't run on iOS, so there's no point in comparing to Apple. For those on Android, NetGuard is open-source, https://github.com/M66B/NetGuard
On iOS, Lockdown firewall is open-source, https://github.com/confirmedcode/Lockdown-iOS
If they advertise "What happens on your iPhone, stays on your iPhone.", these kinds of options must be off by default. Since they are on, I can consider this one and their other statements about privacy lie.
I'm moving my family out of apple photos, self hosted options have come a long way. I landed on immich [0] and a caddy plugin that allows for PKI certificate for account access while still allowing public shared URLs [1]*
There's also LibrePhotos which is packed with features but doesn't have as much polish as immich. They do however have a list of python libraries that can be used for offline/local inference for things like having an image model create a description of a photo that can benefit the full text search. [2]
[0] https://immich.app/
[1] https://github.com/alangrainger/immich-public-proxy/blob/mai...
[2] https://docs.librephotos.com/docs/user-guide/offline-setup
* Haven't actually tried this plugin yet, my weekend project is setting up my caddy VPS to tunnel to the immich container running on my Synology NAS
Immich is amazing, truly great software, and it's oss
It’s still beta though and often has breaking changes that can require updating the server software in sync with the mobile app versions.
i like to self host but i’d never do it for something critical like photos. what if you’re hit by a bus next month? Then your family (probably not tech savvy) need to figure out how to migrate all their stuff to a commercial offering - while dealing with all the emotional stuff after death. it’s just added stress for your loved ones.
100% private is nice but not practical for something like photos in my opinion. unless you have someone in the family that can take over after you’re gone and is very technical.
Sure, I use Ente and have iCloud Photos turned off, but TFA is referring to what happens in the Apple Photos app, which happens even if you have iCloud Photos turned off.
Here's a direct link to the paper that describes this as mentioned in the post,
https://machinelearning.apple.com/research/homomorphic-encry...
The homomorphic code is available in Swift
https://www.swift.org/blog/announcing-swift-homomorphic-encr...
I have concerns about how women would be affected by law enforcement who might use the location data to target abortions.
> I have concerns about how women would be affected by law enforcement who might use the location data to target abortions.
I believe this risk already exists in the form of LE request with court order to see the contents of an iCloud account, such as conversations and photo metadata. Advanced Data Protection exists to eliminate this particular vector - Apple no longer has account recovery information like private keys escrowed, so there's no way they can provide it if requested.
This would not increase such a risk since the network traffic is effectively VPN'd for anonymization, clients send false data periodically to break profiling or behavioral analysis, and the requests and corresponding results are not comprehensible to Apple's server due to the HE.
> I have concerns about how women would be affected by law enforcement who might use the location data to target abortions.
What? How?
police using photos/location to figure out that a woman from Texas went to an address associated with an abortion clinic, and wow Texas has bounties, noooo potential for abuse there.
you've really been shilling apple and even obnoxiously giving multiple replies to the same comment, what's your problem?
2 replies →
The referenced Apple blog post[1] is pretty clear on what this feature does, and I wish the author at lapcatsoftware (as well as folks here) would have read it too, instead of taking the blog post as-is.
Apple has implemented homomorphic encryption[2], which they can use to compute distance metrics such as cosine similarity without revealing the original query/embedding to the server. In the case of photos, an on-device model is first used to detect if a landmark may be possibly present in the image, and then sends an encrypted embedding[3] for the image region containing said landmark, which is evaluated on the server using HE techniques and then the encrypted results are sent back.
I'm sure someone will come along and say Apple does none of these things; in which case, said commenter should probably not use Apple devices since, there is no reason to trust the toggles for "Enhanced Visual Search", and perhaps more importantly, the one for "Advanced Data Protection" either. However, I rarely see any other major company researching ML and HE together, so that alone is a sufficient positive indicator in my eyes.
(It's interesting that this is also being downvoted. Please go ahead, I can't stop you :P but please also write the parts that you disagree with. Thank you.)
[1] https://machinelearning.apple.com/research/homomorphic-encry...
[2] https://en.wikipedia.org/wiki/Homomorphic_encryption
[3] https://mlr.cdn-apple.com/video/HE_Fig_3_PNNS_889f3a279c.m4v
> please also write the parts that you disagree with. Thank you
The problem invoked by the article is that data is being sent back to Apple by default. Saying "it's fine because it's encrypted" and "don't use apple if you're not fine with that" doesn't help.
The post complains about a product that stored sensitive customer content locally now sends that data to Apple, and given the combination of abuse on privacy and horrendous, generalized security failures that we've seen across the industry, those concerns seem genuine. Your comment is very dismissive of these concerns, which would explain why it's being down voted.
Apple also makes a mandatory API call to captive.apple.com from every device with a web view, just about every time they connect to a network.
If someone is willing to take a hardline stance that a vendor should inform the user and require opt-in consent for every type of exchange, they are going to have to run platforms specifically targeted toward that mindset.
You're extensively describing the technical implementation while missing the fundamental issue: Why is Apple enabling this feature by default for what is, essentially, a luxury photo search feature?
Let's break this down:
1. Nobody is questioning whether Apple's technical implementation is sophisticated or secure. It clearly is.
2. Nobody is suggesting the privacy toggles don't work. They do.
3. The issue is about Apple deciding that automatically sending data about users' photos (regardless of how securely) is an acceptable default for a feature many users may never want or need.
Consider the value proposition here: Apple invested significant engineering resources into complex homomorphic encryption and differential privacy... so users can search for landmarks in their photos? And they deemed this feature so essential that it should be enabled by default?
This feels like using a golden vault with military-grade security to store grocery lists. Yes, the security is impressive, but that's not the point. The point is: Why are my grocery lists being moved to the vault without asking me first?
A privacy-respecting approach would be: "Hey, would you like to enable enhanced landmark search in your photos? We've built some really impressive privacy protections to make this secure..."
Instead of: "We've already started analyzing your photos for landmarks because we built really impressive privacy protections..."
The sophistication of the technology doesn't justify making it the default for what is ultimately an optional convenience feature.
> users may never want or need
Are you assuming that Apple did not perform a market analysis when implementing this feature? I think that is unlikely, considering the effort.
Thank you for posting the apple blog post. As usual, it's really interesting research, and it's fascinating to see how they solve potentially gnarly privacy issues.
> Thank you for posting the apple blog post.
The very same blog post is linked in the submitted article.
1 reply →
Everything about what they did was absolutely fantastic and amazing, until they turned it on by default.
That's not the point of the outrage though (at least not for me). They enabled by default a feature that analyzes my pictures (which I never upload to iCloud) and sends information about them to their (and others') servers. That is a gross violation of privacy.
To be clear, I don't care about any encryption scheme they may be using, the gist is that they feel entitled to reach into their users' most private data (the photos they explicitly said they don't want to upload to iCloud) and "analyze" them.
This is the same as that time Microsoft enabled OneDrive "by mistake" and started slurping people's private documents and photos saved in default locations (arguably worse since no one takes pictures with their PC's webcams).
If you really didn't want your photos to be analyzed, would you be using an iPhone? Or any modern smartphone? Google photos doesn't have nearly the privacy focus and no HE whatsoever but I rarely see that mentioned here. It almost seems like Apple gets held to a higher standard just because they have privacy preserving initiatives. Do you use a keyboard on your iphone? You may not have heard but apple is tracking which emojis you type most often [0] and they get sent to apple servers.
[0] https://www.apple.com/privacy/docs/Differential_Privacy_Over...
3 replies →
If the data is encrypted, does the concern still apply?
You bring up the example of Onedrive, but there is no use of e2e encryption or HE techniques there.
18 replies →
I thought less about the data, as much as the metadata, direct and indirect.
Just from memory when the scheme came up in earlier discussion.
The system is essentially scanning for the signature for some known set of images of abuse so that it aims to capture abusers who would naively keep just these images on their machines. (It can't determine if a new image is abusive, notably).
It's conceivable some number of (foolish and abusive) people will be caught this way and those favoring a long dragnet for this stuff will be happy. But this opens the possibility that a hacker could upload an image to an innocent person's computer and get that person arrested. Those favoring the long dragnet will naturally say the ends justify the means and you can't make an omelet without cracking a few eggs. Oh, "think of the children".
Edit: Also worth adding that once a company is scanning user content to try to decide if the user is bad, it makes it that much easier to scan all kind of content in all kind of ways for all kinds of reasons. "for the good", naturally.
That was the CSAM thing that they have announced they gave up on.
This is totally different.
3 replies →
>The system is essentially scanning for the signature for some known set of images of abuse
Huh? The system is scanning for landmarks, not images of abuse.
>people will be caught this way
Due to the homomorphic encryption, I don't think Apple even knows whether the image matches a landmark in Apple's server database or not. So even if Apple put some images of abuse into its server database (which Apple claims only contains pictures of landmarks), I don't think Apple would know whether there was a match or not.
2 replies →
Was this not announced during the iOS18 'Glowtime' event? Gosh. Thanks for the neon! https://www.cnet.com/tech/mobile/everything-announced-at-app...
https://www.internetsociety.org/resources/doc/2023/client-si...
Should I have to watch marketing events to make sure I'm not being spied on?
Maybe just the ones with bright neon "Glow-foo" canary branding..
More on homomorphic encryption here: https://www.swift.org/blog/announcing-swift-homomorphic-encr...
Summary: "Homomorphic encryption (HE) is a cryptographic technique that enables computation on encrypted data without revealing the underlying unencrypted data to the operating process. It provides a means for clients to send encrypted data to a server, which operates on that encrypted data and returns a result that the client can decrypt. During the execution of the request, the server itself never decrypts the original data or even has access to the decryption key. Such an approach presents new opportunities for cloud services to operate while protecting the privacy and security of a user’s data, which is obviously highly attractive for many scenarios."
I don't understand why this feature needs to exist at all. Ostensibly, if someone wanted to use this, they also have location data on, the phone not only knows its location but also which direction it is pointed via the magnetometer.
I understand it is doing feature matching on the image, but if you were taking a picture of the statue of liberty, the phone would already know that from existing signals.
Every time one of these articles pops up about this sort of behaviour but one of the big tech companies, I'm always struck by the differences in the comments you get. I wonder if Google would be getting the benefit of the doubt like apple is currently getting were Google to implement the exact same thing as an opt-out feature?
It doesn’t seem odd to me. I’m more suspicious of Google’s motives and am even more suspicious of companies like TikTok, Amazon, Xiaomi, and Facebook. There’s a privacy spectrum.
I think sometimes the reactions are different because the facts are different. Your counterfactual seems less likely to me than eg Google photos analysing and indexing each image.
A short while back, Google implemented what I call "Constantly Opt-Out". Google Photos frequently prompting you to "backup those photos to ...".
Why does Apple keep insisting that I want anything to do with them after I bought the device?
Let go of the freaking tether.
And let me choose my own photo editing service provider.
You literally can choose your own photo software. You don't have to use Photos at all. Just get another app.
Perhaps it's the legion of kool-aid drinkers that preceded you and influenced their design.
> Why does Apple keep insisting that I want anything to do with them after I bought the device?
Um, because that is what their customers are asking for?
It sounds like Apple implemented a feature and went out of the way to preserve privacy. For most users the correct choice here is to enable the feature by default.
Maybe this feature is to turn this off?
Years ago I went to a concert with some friends. We took pictures standing in front of the stage while the staff was setting up. There were some additional pictures of the show. Nothing in any of the pictures had any information regarding the band names. This was taken with an iPhone 6s, so no "intelligence". In the past year or two I had a new "memories" album. The album title was the artist name that we went to see. At first I thought this was cool, then I was concerned, like, wait, how did they get this information?
My guess, the "intelligence" used the GPS info to pull up the venue and with the date figured out who the artist was. The album was the headliner, the group photo was of one of the opening bands.
I'm still on iOS 17, and I don't see any other way to turn this feature off. I have an option to reset suggested memories, but that's it.
It sounds more like a users are given a choice now, which would be a good thing. If it was enabled pre iOS 18 that kind of makes sense that it's enabled by default now.
If your core concern is privacy, surely you'd be fine with "no bytes ever leave my device". But that's a big-hammer way to ensure no one sees your private data. What about external (iCloud/general cloud) storage? That's pretty useful, and if all your data is encrypted in such a way that only you can read it, would you consider that private? If done properly, I would say that meets the goal.
What if, in addition to storage, I'd like to use some form of cloud compute on my data? If my device preprocesses/anonymizes my data, and the server involved uses homomorphic encryption so that it also can't read my data, is that not also good enough? It's frustrating to see how much above and beyond Apple has taken this simple service to actually preserve user privacy.
I get that enabling things by default triggers some old wounds. But I can understand the argument that it's okay to enable off-device use of personal data IF it's completely anonymous and privacy preserving. That actually seems very reasonable. None of the other mega-tech companies come close to this standard.
iCloud is opt in. This should be too. A lot of people are fine with keeping their photos offline-only and syncing with their computers through a cable.
Making it “private” with clever encryption is their job since Apple wants to sell privacy. They aren’t doing it because they are nice or care about us. Plus, code is written by people and people write bugs. How can you tell this is truly bug-free and doesn’t leak anything?
Ultimately, making it opt-in would be painless and could be enabled with a simple banner explaining the feature after the update or on first boot, like all their opt-in features. Making it opt-out is irresponsible to their branding at best and sketchy to their users at worst, no matter how clever they say it is.
No — users should be the ones to decide if “encrypted on remote storage” is a beneficial trade off for them and their particular situation.
I think there’s some weird impulse to control others behind these decisions — and I oppose that relationship paradigm on its own grounds, independent from privacy: a company has no business making those choices for me.
You are free to use such services if you wish; others are free not to use those services.
I have no idea what apps are sharing with what. On Android network access is so ambiguous. There's such fuzzy wording. Like when you are asked for location permission to use bluetooth. Even perms like file system access. I don't know what this extends to. Have no idea what it is doing. I recently set up a new ipad. I failed to even work out where Photos ended up. Couldn't work out what was backed up, and what wasn't. How to opt out of everything and opt in piecemeal. Whether the OS/gadget had AI enhancements, what they were or are, whether the apps would have cloud access or not. In fact for an apple device it bugged me with dialogs from the get go. And bluetooth kept turning itself back on. I would say I am technically savvy, but I was pretty clueless. I was quite keen to try out some of the AI photo tools. Like find pictures of such and such, but I didn't get that far as confusion abound.
So what can Apple do with this assuming they can send anything not just "landmarks".
What metadata about the image do they get?
Are images from the web or social media or device screenshots accessible too?
Seems at the very least they are getting a database of things, objects and shapes. Can cohorts of users be identified? Useful for ad targeting?
They should be able to determine how many devices have a photo of a particular person. Or have a photo of a piece of art. Useful for capturing wanted people and stolen goods! They should see how many times users take a photo of another phone. When and how many users take a photo of a protest. Which side has which colour. Do people take selfies of themselves when sick with a novel pandemic.
Do people take more photos of dogs at Christmas than cats.
Anything that can be vectorized can be identified. But what data do they get and what could they do with it?
It's like telemetry and database of images.
I think the differential privacy and encryption are meant to help with that.
Apple TOS:
> To uphold our commitment to privacy while delivering these experiences, we have implemented a combination of technologies to help ensure these server lookups are private, efficient, and scalable.
Efficiency and scalability have nothing to do with "upholding one's commitment to privacy". This shows they're insincere.
But, is privacy achievable today? I doubt it. People desperately want (or think they want) features that are opposite to privacy, and if you don't give it to them, they're unhappy. I think Apple's fault is in promising something they can't deliver. (That also goes for Brave, Kagi, Duck Duck Go, etc.)
Scott McNealy famously said "You have zero privacy. Get over it." This was in January 1999. 25 years ago!
> Efficiency and scalability have nothing to do with "upholding one's commitment to privacy". This shows they're insincere.
"private, efficient, and scalable" means "private AND efficient AND scalable". What makes you think they are being insincere about the privacy aspect?
If they said To uphold our commitment to privacy while delivering these experiences, we have implemented a combination of technologies to help ensure these server lookups are private then it would be fine. It would also be a tautology.
When they add that server lookups are also "efficient and scalable", it means that they have had to ponder the privacy aspects with technical concerns regarding efficiency and scalability, and that therefore, privacy is mitigated.
I think a fair reading of this sentence would be: "we provide a version of 'privacy' that we feel is acceptable, within reasonable efficiency and scalability constraints".
They're not going to dedicate a server per customer for example. Would it make sense to do it? No. But it would be honest to say "because of efficiency and scalability limits, the 'privacy' we provide is relative and subject to breaches". (I actually think that's exactly what their wording is trying to say.)
3 replies →
Going off a tangent, I wonder if the market reveals survival bias: companies and products that did respect privacy practice (e.g. by asking explicit permission) were not able to harness enough user data to compete with other bad players, and as a result, any company would eventually end up like Facebook or go out of business.
Sadly privacy is not a marketable feature, or at least it does not have the ROI as Apple originally believed. I feel the only way to level the play field is to reconsider our regulation framework and treat privacy as a fundamental benefit for consumers.
Apple can do this because there are absolutely no significant consequences. In the classical legal world your only recourse to this invasion of your privacy along with falsified advertising is a "class action suit". It is clear from past class actions suits that (remember the batter thing) this does not prevent Apple from doing the same thing again.
The problem is the granularity. How do millions of people recover damages when a corporation knowingly (knows or should know) it is acting to enrich itself in a significant manner in small amounts. Let us suppose that an abstraction of the damage from this offense can be quantified at $1 per customer. A significant question is whether this action has any possible benefit to Apple. If it does not then once notified of this action, Apple would immediately (as in the next update of IOS or Sequoia) remedy this feature.
So step #1 is someone to send an official letter, perhaps with just a link to this article. Or perhaps someone from Apple is reading these comments and can inform us whether they are aware.
Next state is that Apple is aware of this problem (knows or should know) and ignores it. So are you helpless? Or could you file a claim in small claims court seeking to recover $1 for each photo that Apple has unloaded, requesting that Apple delete all photos as well as all data derived from the photos, plus filing fee.
Next state. You comment on this post as to how to improve the process and make it faster and easier. [And if you oppose this idea, could you please explain your position in a way that helps others understand?]
Just ged rid if icloud and apple photos. slow as f*ck, crash all the time once the library get big enough on my mac. damn slow to scoll in the photos app with lot of picture.
I switched to https://immich.app/docs/features/command-line-interface and import all of my photo into it. 1.3TB and it's damn fast with all the face detection and ml features.
Is it possible to search immich images via spotlight like one can from iCloud photos? I’d love to jump ship away from iCloud but spotlight search is legitimately helpful and I use it many times a week. It’s saved my bacon quite a few times too (didn’t realize I had eg a document as a photo or pdf instead of email).
I have 60k photos using 1.2TB and Photos.app feels faster than Google Photos or Lightroom to me ¯\_(ツ)_/¯
It sounds like Apple Photos "phones home" by fetching some indices? It doesn't sound like photo data leave the device, but I think Apple's issue is they should have been more clear about how this feature worked and what data is and isn't transmitted.
If it works how I think it works, I actually don't see an issue here. I don't think anyone here expects there phone to not "phone home" to check for OS updates by default?
It's been doing this since long before iOS 18. I first observed this behavior already on my 1st gen. iPhone SE with iOS 11 or maybe iOS 12. Every now and then, especially after having taken new pictures with the camera, I'd see the network activity spinner get going whenever starting Photos - and I've never enabled iCloud backups, but rather explicitly always disabled it everywhere I find a setting to do so. With Wi-Fi it'd be a short one second affair, and on cellular it'd take a few seconds more. I also observed the duration of the activity to vaguely extend with the number of new pictures taken between starting Photos. Curiously, when disabling cellular data permission for Photos and having no Wi-Fi connected, the network activity spinner would still pop up and just get stuck indefinitely without ever finishing whatever it's reporting.
My gut feeling has always been that it's returning digested machine learning feedback for Photos' facial/object recognition.
(add.: since moving from the iPhone 8 and iOS 16 to a modern iPhone and iOS 17/18 this behavior has gone away - at least the network activity spinner doesn't gossip anymore...)
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
Couldn't they just make a k/v database that has coordinates of landmarks and then just pass the lat/long from the picture's EXIF in an API call for lookup in said DB when the user wants to make use of this feature? Isn't this part of why we have geotagging?
Reading comments here feels like being on Twitter, Reddit and 4chan combined - a lot of people not listening to each other.
What happened to old HN?
I've noticed that too. I think it depends on the topic. Many are biased when it comes to Apple (me too). Some even defend their misbehaving. My emotional reaction to Apple misbehaving is usually anger because I somehow feel disappointed, even betrayed by Apple. "Apple, how can you build such gorgeous hardware and still act so unethical?". This is of course irrational: Apple is a company that does everything to grow.
My view is that a large group of people interested in building companies, tools, etc moved on and only come back when there’s a PR issue that pushes then to comment. What’s left behind is basically the same old /. esque crowd from the internet of old.
Too mainstream, regression to mean.
I think the user should be prompted to enable new functionality that sends data to the cloud. I think there should be a link to details about which data is being sent, how it is being transmitted, if it is stored, if it is provided to 3rd-parties, and what is being done with it.
Maybe this is exactly what the GDPR does. But it is what I would appreciate as a user.
I have seen how sending metrics and crash dumps from a mobile device can radically improve the user experience.
I have also seen enough of the Google bid traffic to know how much they know about us.
I want to enable sending metrics, but most of the time it's a radio button labeled "metrics" and I don't know what's going on.
Apple also does this thing where, after a major version update (or, seemingly, random incremental updates) it'll just go ahead and reset your app settings for you. So, make sure to check every single privacy option once a month or so, or just give in and let Apple do whatever it wants.
> It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations.
It's a recurring theme in the modern IT industry that the user can somehow never trusted to take their own responsibility for anything. Like, can I please not have the "added security" on my Android phone where it would sometimes ask me for the pattern instead of the fingerprint? I know my own risks, thank you very much, Google. Same for macOS occasionally making me enter the password instead of letting me unlock with touch ID. Same for a bunch of other random software treating me as a complete technologically illiterate idiot with no way to opt out of this hell.
Remember how our devices used to serve us, not the other way around?
It's the rise of authoritarianism, and its appearance in IT mirrors what's happening in the rest of society too.
I'm genuinely surprised that there are many software developers that defend Apple products as great tool for privacy even though we are not able to inspect their code that is closed.
Am I missing something or is it just hypocrisy?
It's just hypocrisy. I don't know how people can defend Apple as a privacy company when they phone home more than their Google counterparts.
People will say that at least Apple don't have a vested interest in their data since they are not an advertising company like Google or Meta. But Apple collects an absurd amount of data, it is phoning home every second. Some Apple traffic is hard-coded to go through VPNs.
Apple has already experimented with ads in Maps. Shareholders demand infinite growth, so it is only a matter of time before Apple dives into ads themselves.
The only valid point is that Apple does protect your privacy – only towards other companies.
I switched to Apple around 3 years ago because I don't like having my data coupled to Google. But I am regretting my decision, I will probably switch back to Android + Linux instead of IOS + Mac at this rate.
Degoogled Android (like LineageOS) is real thing that many use on daily basis.
Wow, this was even enabled on our otherwise fairly locked down corporate phones... Normally these features are both disabled and greyed out via "group policies" or whatever Apple calls them.
Related:
This is on by default for all apps and it “Allows Siri to learn from how you use [an app] to make suggestions across apps”. I found no global setting for all apps.
More details: https://support.apple.com/en-mn/guide/iphone/iph6f94af287/io...
We need to go back to local-first software. I'd be happy to pay premium for a device, which is truly my, without big tech sniffing all my data or deciding what can I do with it
Anyone using a hardware firewall with apple devices explicitly balcklisting Apple's IP address range? If so, any interesting issues to watch for?
The big question for me is what data contributes to the global index. I would definitely be unhappy if my own photos (pixels + GPS location) were being added to this global index. For example, if someone takes a picture of my house, could they find the location (reverse image to location lookup) using this service? Does anyone know what comprises this global index?
Don't be too worried about what they collect per se they all do and you mostly won't know. Let's worry about how they manage and use it. When it comes to Apple specifically - homomorphic encryption is "common" and well done - it how do they use this encrypted data is the key...
Apple decided to siphon user photo derived data without prior consent. The purpose, capacity and risk of said data being in others’ hands we are expected to take their word for. All while not informing users in advance or trusting users to themselves decide to start the siphoning. By doing this they have violated some users trust.
> Remember this advertisement? "What happens on your iPhone, stays on your iPhone."
1. Take the biggest downside of your product
2. Advertise the opposite
3. Profit
I have replaced “Apple Photos” with photoprism. I am happy so far.
Related blog post: https://www.convalesco.org/notes/2024/09/29/setup-photoprism...
I kept telling people here and on Reddit that Apple is using the whole idea of "Privacy First" on Apple devices as a marketing gimmick. The company has began introducing bloatware into their product and rivaling Windows bloatware.
Yet, people perception on Apple bloatware here seem to be, "that's fine".
> From my own perspective, computing privacy is simple: if something happens entirely on my computer, then it's private, whereas if my computer sends data to the manufacturer of the computer, then it's not private, or at least not entirely private. Thus, the only way to guarantee computing privacy is to not send data off the device.
+1
You can't even turn on and use a new apple product unless you phone home and ask apple for permission. I'm not sure why this is newsworthy. Apple products are known to phone home more than just about anything else. It would be more surprising if apple photos didn't phone home.
I don't mind the whole use of ML based features even if it requires elaborate schemes to run some computation in the cloud in some privacy preserving manner.
However I do see that Apple marketing campaign tried to fool people into thinking somehow non Apple are not private and only Apple provides privacy.
This obviously requires location services to be enabled. Why does this need ML? Hit a GIS lookup API with the lat/long of the photo.
Install proxyman, you can find some interesting things about your favorite beloved iOS. It is not just photos app. They track me, I track them.
Worth noting that you need iCloud active in the first place to opt-out of this feature, so it is still opt-in, since it is only ever defaulted to being on for users who proactively signed into iCloud.
This is not true.
Ah yes, you're right. I spoke to soon. I just checked on the latest Beta simulator and, sure enough, it is checked, with a fresh install and no iCloud account.
At least that proves the service is accessible anonymously (i.e., without an identifying iCloud token)
Most Apple Photos users keep their actual photos on Apple's computers. There are few people who should react to this.
Sadly, I'm one of them.
So it sends your photos to be indexed on Apple servers. Turned on by default.
This is probably done to compete with Google Photos which has a great photo search by word feature.
With that said, Apple can use whatever privacy measures to protect user data. But at the end of the day, a subpoena can easily force them to hand over data.
The best privacy measure is to just not have the data. I guess indexing photos offline in phone is not very feasible yet.
> This is probably done to compete with Google Photos which has a great photo search by word feature.
Apple Photos is already capable of searching your photos by word/content fully offline on-device.
Google Photos is unable to search your photos by word/content fully offline on-device. Hell, it can't even search by file name if you don't sync your photos to the cloud!
I don't think Apple has to worry about Google at all.
Neither is Apple, FTA.
> I guess indexing photos offline in phone is not very feasible yet.
There has been a form of on device indexing since at least iOS 12. My understanding is it performs a basic indexing typically overnight when the phone is charging and allows one to perform searches like "dog" or "car" and pull up matching images.
https://support.apple.com/guide/iphone/search-in-photos-iph3...
That's nice but clearly not enough to compete with Google Photos.
Hence why Apple started sending photo hashes to their servers for indexing by default.
There's only so much offline computing power.
It does not send your photos to be indexed on Apple servers.
it literally says it uses global search index on the label below the check mark. It seems more than likely that (now or at least in the long run) they will use user data to enhance this index.
2 replies →
Also, by the time you find that button to turn it off, Apple already has the info it wanted...
At least google told me on my phone in a pop-up that it would be doing something like this.
whats the 'feature' in the app that requires sending a hash off device?
It's called "Enhanced Visual Search" and they described it like this:
> Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest.
I guess it's useful if you get very lost and don't recognize the pyramids right in front of you, and forgot you have a GPS in the phone.
Apparently, if you take a photo of the Taj Mahal, it will find photos other people took of the Taj Mahal so you can feel inadequate.
This is exactly the kind of feature I have this app for.
I'm disappointed in the discourse around Homomorphic Encryption and Differential Privacy here. As someone who briefly studied adjacent subjects these tools excite me more than they scare me.
We trust TLS with our SSNs and CC numbers, hopefully one day we can trust secure multiparty computation too.
>We trust TLS
Is this really an apt comparison? I understood the trust in TLS to be built on open RFCs and implementation stacks. Even then, whenever I send private data, I take specific steps to verify I am using that trusted stack. That is not the experience described in the article.
> I take specific steps to verify I am using that trusted stack
I would be very interested to hear what these specific steps are. How do you make sure that this TLS stack really does implement the RFC? How do you know that each connection is indeed encrypted, and it doesn't start sending plaintext after, say 30 days of usage?
Let’s step back and go through the thought process of the team that’s implementing this feature. If they leave the feature disabled by default most likely casual users will never use it because they won’t be able to find the setting buried under all the menus. Thus after adding all their privacy layers the team felt that it should be safe enough to enable by default while remaining true to the company’s ethos of privacy.
Now what would you have done differently if you were in charge of rolling out such a feature? While I don’t like Apple phoning home without my consent, in this case my photos are not being sent to Apple; only anonymized hashes used to match well known landmarks.
The author of the article goes on to show his bias against Apple with phrases like “You don't even have to hypothesize lies, conspiracies, or malicious intentions on the part of Apple to be suspicious of their privacy claims” or “Apple's QA nowadays is atrocious.”
Or this rich quote “I never wanted my iPhone to phone home to Apple.” What smartphone or computing device never communicates back to their company servers? Even when I use open source libraries I have to communicate with repositories to check for dependencies. Does the author hide every online transaction he makes? Never using email or text messages or cloud services which will leak his private information? Unlikely. He just wants to grind his axe against Apple.
So let’s step back and look at it reasonably and see if Apple is trying to erode our privacy with this feature. I personally don’t see this particular feature as being harmful, but I will thank the overzealous author of bringing it to my attention and I’ll be disabling the feature since I don’t need it. This feature is no where near as invasive as the CSAM detection tool that was proposed, which did warrant critical discussion.
Let’s let the team, undoubtedly one of many with the unenviable task of making Apple Intelligence relevant, who rolled out the feature get their yearly bonus and move on to discuss more enlightening topics such as as the merits of keeping tabs on the urination habits of the crew on the ISS via the Mac menubar.
> The author of the article goes on to show his bias against Apple
My bias against Apple, as someone who exclusively uses and develops software for Apple products? I've been a loyal Mac user for over 20 years.
It's so ridiculous when people make an accusation like this. For better or worse, I'm married to Apple platforms.
If Apple can prompt me 4700 times after I’ve said no each time to enable Apple TV and Apple Arcade and Apple Fitness subscriptions, they can prompt users to enable a new feature if they wish.
A silent default opt-in erodes trust!
Now "What happens on your iPhone stays on your iPhone" seems like it deserves the Lindt chocolates defense: "exaggerated advertising, blustering, and boasting upon which no reasonable buyer would rely."
Tbh whilst this should be improved it is more like those who lives and works in free democratic countries with market access by and large complained about their imperfect capitalism, undemocratic and not social enough country. May be it is.
But I do not think so by and large.
Keep on fight and Lee on pushing. At least Apple has a slogan you can push on.
looks like its not even available on EU, I'll call that a win
Where are you? I'm in Poland and I just disabled it both on phone and tablet - the setting switcher is "conveniently" placed at the bottom of Photos settings page
Womp womp...
Who would have thought the amoral money making machine would want to make more money? :-)
Got to feed their piss poor "Apple Intelligence", somehow.
Whatever embeddings and search history you make on your phone that is "generated locally" goes straight bak to apple by default.
They make it impractical for anyone to NOT use iCloud.
It's all reality distrotion field marketing bs.
It is frankly nothing short of amazing that Apple ships things like homomorphic encryption, and differential privacy, and client-side vectorization, and encrypted vectors, at the scale that they inhabit... and they still get a bad report card back from consumers about privacy.
Comparing Apple and Google, or Apple and Microsoft, it seems to me that Apple's track record on these issues is actually not as bad as public opinion might suggest. Meta doesn't even make the list for comparison, and neither does Amazon.
It makes me wonder if picking privacy as a strategy is workable in the first place. People trust TLS; people use banking apps on their phone now without thinking. I remember in 2008 or so when people still didn't quite trust SSL.
I'm not sure if Apple will be able to bridge the gap here, though, if _all_ of their competition simply chooses not to ship those features. Do customers know the difference? Do they... care? In theory they want their data to be private, yes. But if they are not willing to educate themselves to perform their counterparty obligation in fulfilling "informed consent," there may be no action Apple could take to avoid catching the same bad rap everyone else does.
Consumers seem to think well of their pro-privacy standpoints, even if the devil is in the details with regards to how effective it might be.
The people giving “poor reports” are often the hardcore tech users (or LARPers) who grew up in a different era and mindset and are slowly being shoved out.
How did they do it? The idea for homomorphic encryption has been around a long time, but as far as I am aware the time+space requirements have found to be tremendous, thus rendering any serious implementation of it impractible.
If they managed to do it, they should open-source the code. If they made a research breakthrough, they should publish. Doing either of those things would give me the peace of mind that I need.
https://www.swift.org/blog/announcing-swift-homomorphic-encr...
https://github.com/apple/swift-homomorphic-encryption
https://machinelearning.apple.com/research/homomorphic-encry...
https://www.apple.com/child-safety/pdf/Apple_PSI_System_Secu...
1 reply →
> It is frankly nothing short of amazing that Apple ships things like homomorphic encryption, and differential privacy, and client-side vectorization, and encrypted vectors, at the scale that they inhabit... and they still get a bad report card back from consumers about privacy.
Personally, I don't shy away from criticizing Google, but that's not the point. Apple makes big claims about their privacy practices that neither Google nor Microsoft make, it would be bizarre to hold Google or Microsoft to claims and standards that Apple set for themselves.
> Comparing Apple and Google, or Apple and Microsoft, it seems to me that Apple's track record on these issues is actually not as bad as public opinion might suggest. Meta doesn't even make the list for comparison, and neither does Amazon.
> It makes me wonder if picking privacy as a strategy is workable in the first place. People trust TLS; people use banking apps on their phone now without thinking. I remember in 2008 or so when people still didn't quite trust SSL.
> I'm not sure if Apple will be able to bridge the gap here, though, if _all_ of their competition simply chooses not to ship those features. Do customers know the difference? Do they... care? In theory they want their data to be private, yes. But if they are not willing to educate themselves to perform their counterparty obligation in fulfilling "informed consent," there may be no action Apple could take to avoid catching the same bad rap everyone else does.
I've said this elsewhere, but what I really dislike about Apple's strategy regarding privacy is that they treat privacy as a purely technological problem that could be solved if only we had better technology, but they ignore that a huge component of why users have been subjected to so many flagrant privacy violations is because they have zero visibility and zero real control over their computing experiences. Apple would very much like to retain their iron grip on what users are allowed to do on their platforms, as they make a ton of money off of their control of the platforms in various ways, so they have a huge incentive to make sure we're all arguing about whether or not Apple is better or worse than Google or Microsoft. Because sure, I do believe that if we held Google to the same privacy standards that Apple currently has, it would probably kill Google. However, if Apple and Google were both forced to give more transparency and control to the users somehow, they'd both be in a lot of trouble.
Despite all of this effort, I think that user trust in the privacy of cloud computing and pushing data out to the internet will only ever go down, because attacks against user privacy and security will only ever continue to get more and more sophisticated as long as there are resourceful people who have a reason to perform said attacks. And there certainly always will be those resourceful people, including in many cases our own governments, unfortunately.
“Ask App Not To Track” would like a word
8 replies →
Lovely.
And this blog post is how I find out the Photos app mysteriously turned iCloud Photos back on. What the fuck? At least "Keep Originals" was still set.
Holy crap! Enabled by default! Thank you for letting everyone know.
“Enhanced Visual Search in Photos allows you to search for photos using landmarks or points of interest. Your device privately matches places in your photos to a global index Apple maintains on our servers. We apply homomorphic encryption and differential privacy, and use an OHTTP relay that hides IP address. This prevents Apple learning about the information in your photos. You can turn off Enhanced Visual Search at any time on your iOS or iPadOS device by going to Settings > Apps > Photos. On Mac, open Photos and go to Settings > General.”
I agree with you 100. I think Apple is doing a couple things fora couple different reasons (all of which I bother me) yet some worse than others It's because it's a Triple (or more) Monetary Win Business Strategy. I'll explain. Has anyone noticed how Apple keeps telling you your storage is "almost full" then wants you to upgrade to ICloud+? Because After selling that expensive phone they don't want to wai to squeeze more money out of you. They set ALZl defaults to open and upload to iCloud, so if you don't change them you are Icloudkng EVERTHING. Which fills up. Plus certain things you can't NOT iCloud bc you don't have a choice. What takes the MOST space and is most important to most people? Photos/Vi. So -) they auto default to upload/iCloud all plus your photos. Then tell you you are out of space and ask you to purchase more and more storage for cost each month from $1-$10.
But - ever tried to DOswLOAD your photos back to phone, and downgrade your storage? You can't. I'm almost impossible. The various instructions online don't work. You can do it with dedktop or laptop but even then they will try to reload and further even THEN there are ALZl kinds of bugs and glitches to getting photos back locally only.
So first part - they set all to upload and make local only imposssible and downgrading storage impossible. I had to take screenshots and file fraud claims to get them to downgrade m and stop charging me 9.99 and 1.99 each month.
But even then. I'm pretty sure they ARE keeping all those photos we supposedly download back OFF iCloud and don't want their storage for AND of course everyone else photos and videos--Why? Second part of strategy (my opinion) - bc they are using ALL our collective photos and videos to Train Large AI models and computers and to run simulations, to prepare for future robots that are more like like, etc. yes U sound a bit conspiracy theorish - but it IS what they are doing! Plus aggregating all the data itself within our devices in photos/videos and other meta data - they can also run that through and sell it, store, make profiles on us, profilers on regions and correlations, etc - all of which is sellable now to various companies AND will be sellable in new ways in the future. So they benefit triple on us. I think there should be federal laws saying -1) All settings must be defaulted to highest privacy (ie closed) and 2) requiring Intentional option in, 3) Plain explanations of what "features and settings do, or how could be used, with clear warnings and subject to Lawsuits for fraudulent inducement and invasion of privacy if those descriptions are understandable by an ordinary reasonable person - NOT an ordinary TECHNICAL engineer. If they can't explain what it does to an ordinary person, it probably is doing stuff that ordinary person WOULD not consent to. And 4 and 5) of new federal laws I wish there - 4) they can't reset or change your privacy settings to make them MORE open after each update, and again req opt-in not opt-out, 5) anyone who does Opt-In - should get metrics of all data disclosed - with a price per amount and data type - PAID back to them. So for example - there should some Global Standards of Data Types - Levels - some more valuable than Others. EXAMPLE/Hypo If it collect "100MBs/amount" of Valuable Level 1 Data, which is Valuaed at $.005/MB then it owes you 100x.005= 5cents. And so on. There should be classifications of data, with Intl standards on the Value of that DATA, and if you did OPT-IN, you get an accounting each month of what data and how much of it did THEY collect, and theyOWE you for collecting it. Bc let's face it - how are they building the powerful computers and AI things they will trillions on? It's on us. Our data. How are making billions in sass and sales- on us, our data.
As it stands NOW they ought to pay us even for data we don't opt-in to but try to Opt out of and can't!! And for all past data they've collected. And for my own time, and everyone else's time - Each month when a new update comes out - THAT EACH TIME resets all those settings I spent hours to reset back to close - and to research those hundreds of "Feature Flags" to find out which are good, which probably are NOT good and which Are in between or who know so roll the dice either way.
I set all settings to closed bc they auto default to everything open, everything sharing, Bluetooth on, public WiFi on, etc everything "uploading" to iCloud. It takes me HOURs. The Accessibility commands and options alone take me an hour to reset. Yet - even when it's turned off - sometimes my phone turns back on something even without updates bc of bugs of hidden features Apple isn't disclosing. Yet after each update all those settings I spent hours on on phones and Mac and iPad, etc- they default BACK to open. And I spend hours again. And it's taking longer each time, and updates while good for other things- for this it sucks, and updates are WAY more frequent. Makes me crazy. I'm not a hacker or a tech like y'all probably are. So I apologize for my long rant by a novice such as myself. I'm just a normal person who likes to read and research and teach myself things so even though I'm not technical I doread a lot and so I'm not an idiot either. I've been a Mac user for over 20 years. Use to an @mac, then @me, and of course now @icl - things starting really going to shit and having sync/corruption issues and bugs, etc in 2019, got way worse in 2021, and now it's all the time. It's so frustrating though bc everything - is Mac-it's the invested infrastructure of our lives - to use an analogy: it's like Apple/Icloud is a Hotel California. It's used to be a nice hotel though. Now it's a Zombie Apocalypse Hotel California. But there aren't any other "hotels" out there that are any better either I'm afraid.
apple: we respect your...
LOL No you don't
[dead]
What “places” would Apple be able to match that aren’t either obvious or irrelevant? (e.g. “The Grand Canyon” is obvious. The interior of a home is irrelevant.) Maybe the interior of a restaurant?
I hate to jump to conspiracy theories, but the mechanism seems like a more effective version of their prior CSAM scanning scheme[1]. Best part? It doesn’t even require iCloud usage.
1: https://www.macworld.com/article/1428633/csam-photo-scanning...
I am on the latest iOS, just discovered this setting, it wasn’t turned on by default.
Mine was turned on by default, I just checked.
Did you upgrade from a previous version?
Maybe the new setting checks a certain combination of other settings first, like:
Mine was also enabled by default.
I have Photos shared library set up, one of the members of the library is a child account — I wonder if that’s got anything to do with it.
odd, i'm also on latest but it was enabled
[flagged]
> if you use iCloud Photos (only that applies)
It's enabled by default for most people, whether they use iCloud or not.
You cannot use iCloud Photos without iCloud. Still not seeing the issue - anyone who cares about their privacy would just turn it off, problem solved.
[flagged]
Oh I dunno. Your comment made me chuckle and I’m largely on the fence on this one but it doesn’t seem too outrageous to say that maybe this is a slight invasion of people’s privacy without asking them to opt in.
Apple: Does backflips to protect your data
Apple's competition: Steals your data outright
Most of this thread: How could Apple act this way
You're kidding yourself if you think apple is in the business of protecting your data beyond marketing itself as such. Just like all it's competition it's in the business of using your data like them.
> You're kidding yourself if you think apple is in the business of protecting your data beyond marketing itself as such.
It is entirely possible to create a business where others ignore user demands, like, say, privacy. Apple does not have a massive advertising incentive to process user data as others do. I am not a huge Apple fan (typing this from my Windows machine), but I am not ashamed to admit that some companies are good at some things, and others at other things (insane idea I know)
Apple does not have to homomorphically encrypt your data. They do so because they have "made privacy a feature," as they say in their marketing. You can also just read the research papers or code if you want, since they have published those, too. Have you read those?
6 replies →
> Apple: Does backflips to protect your data
That's what their marketing says. I'm not sure how far you can actually trust that. They're an ad and media company as well.
I am merely comparing homomorphic encryption, client-side vectorization and ML, and so on, to not doing those things. Nothing to do with marketing. Read the manual
2 replies →
Apple doing the best in class job for privacy and you guys would rather go after it vs google, meta and others who have significantly much massive issues?
I wonder how much of the people that are being upset about an option that recognizes landmarks and that is build in a privacy minded way are using chatGTP, windows, google or socials
The writer of the post admits that he doesn’t understand the very tech lite explanation of Apple nor read about Apple AI in general and the way it was setup. A lot of people are upset that (maybe, but unlikely) Apple knows that you made the 10 billionth photo of the Golden Gate Bridge. But continu using all sorts of services, living in sec camera heavy cities etc.
Not an apple fan but from all big tech corporations they have the least amount of interest in mining user data from a business perspective
>>Not an apple fan but from all big tech corporations they have the least amount of interest in mining user data from a business perspective
I think it's more that they realised they couldn't beat google at that game and hoped to see personal or governmental push back like GPDR in the EU that would then give them an advantage. Which would then give them an advantage. The day they stop having that belief they will invade your privacy.
You're ignoring the part where it's opt-out.
We're way passed the point where things are accidently opt-out instead of opt-in, if you want to dig the point, it's something you should touch upon.