Comment by Quothling
10 days ago
> wrap an external tool in a privacy architecture
Wouldn't it be more accurate to call Apple's architecture data protection rather than privacy? As an European citizen in a post Snowden world I would be surprised if any of my data on Apple services was actually kept private from the US government, and Apple certainly wants to own a lot of data/metadata about you. Gotta have Siri listening for carplay and so on. I would aboslutely trust Apple not to sell my data as a commodity though.
> If Apple handles the Google-Apple boundary right, this will be an elegant move on their part, otherwise it will feel like Apple Intelligence with a just a privacy-polished frontend for Gemini.
I'd say this is spot on. At least if what Microsoft is doing with Copilot Cowork is anything to go by. Cowork is not a privacy-polished as much as it's an Enterprise compliant polish to make Opus 4.8 run "safely" in your enterprise organisation. So far Microsoft is winning the AI war in non-tech enterprise with this, especially here in the EU. If Apple manages to do this for the private market that will be great for them.
I'm not personally sold on what an AI should do on my phone though. I use a lot of AI professionally, but I haven't even turned on Bixby or whatever the Samsung AI is called.
From an EU perspective, Microsoft is doing data protection, Apple is doing data privacy.
Microsoft's approach to data is basically "we promise nobody else but you and your government can access it, we can but we pinky swear we won't." This promise is mostly enforced at the legal layer and through legal consequences, not technical safeguards. If they think they can get away with it (or are forced to get away with it by the US government), there's nothing stopping them from using your data in whatever way they want.
When they can, Apple designs their systems so that they physically don't even have the capability to use your data, even if it's processed on their own servers. They're not privacy maximalists like Signal is, they care more about user experience, but they do aim for the highest level of privacy you can get while still having a good experience, and when they do need to make sacrifices, they typically let you opt into the privacy features if you really want to.
I'm far more inclined to believe that Microsoft is secretly (or not so secretly) collaborating with the US government than that Apple is.
There has been anecdotal statements/blogs from Apple employees about the data privacy. They have said building some internal capabilities or user facing features are extremely difficult or impossible because they aren't able to access user data at the level required.
From the information presented, the privacy case is not that your data is only accessible to you (which arguably can have a backdoor) but that the data is NOT stored at all, so it's not possible to build a backdoor. I know there are probably other ways around it, but it's my understanding is that no data is kept on any server when the response is sent back to your device
I don't think the Snowden allegations were about employee access to user data.
Do you have any examples?
4 replies →
> Wouldn't it be more accurate to call Apple's architecture data protection rather than privacy? As an European citizen in a post Snowden world I would be surprised if any of my data on Apple services was actually kept private from the US government, and Apple certainly wants to own a lot of data/metadata about you.
Your conception doesn’t seem to match PCC at all. The whole point of it is that nobody can access the data, not even the people running the servers.
https://security.apple.com/blog/private-cloud-compute/
I don't trust a single US tech company to keep my data private from the US government. Maybe I need a tinfoil hat, but I don't feel like I'm unjustified in this based on the history going back to echelon. Not that this is a particular jive at the USA, my own government (Danish) actively pushes for mass surveillance and non-functional e2e encryption.
There is still a difference though. Google will sell my data and use it for all sorts of things. Though I've obviously accepted that since I have had a Samsung flip phone since Apple made their iPhones too big for my pockets.
This part of their requirements for how PCC is architected directly addresses your concern:
“Verifiable transparency. Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises. We already have an earlier requirement for our guarantees to be enforceable. Hypothetically, then, if security researchers had sufficient access to the system, they would be able to verify the guarantees. But this last requirement, verifiable transparency, goes one step further and does away with the hypothetical: security researchers must be able to verify the security and privacy guarantees of Private Cloud Compute, and they must be able to verify that the software that’s running in the PCC production environment is the same as the software they inspected when verifying the guarantees.”
17 replies →
It’s a fair concern, but the only way to reconcile a belief that Apple is sharing data from PCC with anyone (including themselves) is to assert the whole PCC thing is a massive fraud.
Which it could be, but given both breadth of claim and Apple’s strong incentives not to be caught lying about something so massive, I’d want something more than vibes to take the idea seriously.
There's no guarantee against data exfiltration, because the data leaks happens through tool calls, which are not made from the PCC, but from your own device.
E.g. "the user asks if their Bitcoin private key is unique, let's make a web search".
Combined with prompt injection attacks, it's quite easy for an attacker to craft a prompt which sends your private data through any supported tool call (web search, database search, email, app APIs, etc.). Everything is wide open for the attacker / or yourself accidentally to exfiltrate your data.
That doesn’t make sense in this context – the point of PCC is so you know somebody isn’t snooping on your information when you send it to the servers. The person I was responding to seemed to think that Apple would be looking at that information.
1 reply →
Which is a good point. set a Bitcoin wallet private key in an obvious place on your system, and then setup a monitor (on another system) to notify you if its contents gets stolen.
Doesn't prevent the exfiltration but at least you'll know when it does.
And we have to believe that it's not backdoored because they say so? That's incredibly naïve.
No. I provided the link so you could read more about it.
2 replies →
> I use a lot of AI professionally, but I haven't even turned on Bixby or whatever the Samsung AI is called.
I know this was just a small aside, but man do I hate Bixby and other phone AIs. They are so frustratingly difficult to turn off, and turning them on accidentally is as simply as holding the wrong button for a few seconds, such as when your phone is in your pocket. Very frustrating design.
I never turned Bixby on, so it never really bothers me except for when I update and it want me to accept something which I decline. I turned the button off, I forgot what I switched it to but holy hell was that annoying.
I never turned Bixby on either, but on my phone is comes pre-installed and pre-turned-on, and you can't uninstall it, only "deactivate" it.
I just want to be able to turn things off without affecting completely other unrelated things. Like Siri and Carplay, makes no sense I need one activated to use the other, just a trick to get people to avoid disabling Siri.
> I would be surprised if any of my data on Apple services was actually kept private from the US government
Outside of law enforcement having a warrant, Apple's efforts against CSAM, or their Chinese data centers, I've not heard of Apple doing any of what you assume in a post-Snowden world. iMessage is supposed to be end to end encrypted, and there was a few years ago that whole scandal where Apple wouldn't unlock a literal terrorists cell phone for the FBI.
The FBI had to reach out to... a third party to unlock the phone (I forget the name of the firm that did it - Cellebrite maybe?) for them, what's funny is they spent a lot of money on it, when the rest of the world pointed out that the very specific iOS version in question had known vulnerabilities they could have found online for free (or cheaper?).