Comment by Quothling
10 days ago
I don't trust a single US tech company to keep my data private from the US government. Maybe I need a tinfoil hat, but I don't feel like I'm unjustified in this based on the history going back to echelon. Not that this is a particular jive at the USA, my own government (Danish) actively pushes for mass surveillance and non-functional e2e encryption.
There is still a difference though. Google will sell my data and use it for all sorts of things. Though I've obviously accepted that since I have had a Samsung flip phone since Apple made their iPhones too big for my pockets.
This part of their requirements for how PCC is architected directly addresses your concern:
“Verifiable transparency. Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises. We already have an earlier requirement for our guarantees to be enforceable. Hypothetically, then, if security researchers had sufficient access to the system, they would be able to verify the guarantees. But this last requirement, verifiable transparency, goes one step further and does away with the hypothetical: security researchers must be able to verify the security and privacy guarantees of Private Cloud Compute, and they must be able to verify that the software that’s running in the PCC production environment is the same as the software they inspected when verifying the guarantees.”
They do this by allowing you to download all of the components (minus data cryptexes containing the model weights) and run it on your own Apple silicon chip (you can put your computer in recovery mode and use csrutil to enable research guest operating systems)
I think what is concerning is that they are expanding into Google Cloud and NVIDIA to run with it too with their versions of confidential compute, which if I remember correctly are not as well verified as Apple PCC and a little harder for researchers to get their hands on.
Apple uses a key ceremony process where no single party has access to all the keys required to sign hardware, meaning in theory they can’t just sign malicious hardware. However, I’m not sure how Google and NVIDIA play into this and I don’t think they’ve provided much detail on it. I think it seems a little rushed to get the features out since they fucked up with initial Apple Intelligence release.
From my understanding of the architecture, Apple and Google have basically developed a fork of Gemini that is built to run on Apple's PCC. There is no data being sent to any Google servers.
From this MacRumors article:
"The new architecture centers on Apple Foundation Models co-developed with Google, which Apple says are adapted to run both on-device and on servers through its existing Private Cloud Compute infrastructure."
And
"The company reiterated that Apple Intelligence relies on on-device processing and Private Cloud Compute, with a promise that user data is only used to execute the immediate request and is not accessible to Apple or third parties. Apple added that outside experts can verify those privacy guarantees "at any time.""
5 replies →
What does verify mean?
Can they verify the private cloud is completely immune to nationstate actors, has no zero-day vulnerabilities, is completely bulletproof in a court of law and can never be compelled to secretly share info with government(s), etc?
I think the users fear here is real. "We did good due diligence at the consumer level" and "we're completely immune to nationstate hackers and clandestine legal cases" are very different things.
You should read the paper.
Like any good security paper, it doesn’t assert immunity to particular parties. Instead, covers things like how PCC attests that the running software image is identical to the publicly-available, forensically-studied one.
Fear is real for sure, but don’t let fear be an excuse to lose rigor in thinking.
7 replies →
How are security researchers going to have access to the Nvidia GPUs that will be running this?
It’s a fair concern, but the only way to reconcile a belief that Apple is sharing data from PCC with anyone (including themselves) is to assert the whole PCC thing is a massive fraud.
Which it could be, but given both breadth of claim and Apple’s strong incentives not to be caught lying about something so massive, I’d want something more than vibes to take the idea seriously.