Comment by Sevii
3 days ago
Apple's goal is likely to run all inference locally. But models aren't good enough yet and there isn't enough RAM in an iPhone. They just need Gemini to buy time until those problems are resolved.
3 days ago
Apple's goal is likely to run all inference locally. But models aren't good enough yet and there isn't enough RAM in an iPhone. They just need Gemini to buy time until those problems are resolved.
Phones will get upgrades, but then so will servers. The local models will always be behind the state of the art running on big iron. You can’t expect to stand still and keep up with the Red Queen.
That was their goal, but in the past couple years they seem to have given up on client-side-only ai. Once they let that go, it became next to impossible to claw back to client only… because as client side ai gets better so does server side, and people’s expectations scale up with server side. And everybody who this was a dealbreaker for left the room already.
Apple thinks they can get a best-of-both-worlds approach with Private Cloud Compute. They believe they can secure private servers specialized to specific client devices in a way that the cloud compute effort is still "client-side" from a trust standpoint, but still able to use extra server-side resources (under lock and key).
I don't know how close to that ideal they've achieved, but especially given this announcement is partly baked on an arrangement with Google that they are allowed to run Gemini on-device and in Private Cloud Compute, without using Google's more direct Gemini services/cloud, I'm excited that they are trying and I'm interested in how this plays out.
You're excited together with a handful of other privacy enthusiasts on HackerNews.
I would think for the vast majority of users out there this is not a concern at all.
Apple until now failed to even get the basics done and make Siri smart, despite marketing "Apple Intelligence" as the core feature of 2024's iPhone.
1 reply →
Given the snowden leaks, i think it’s naive to believe that any data that leaves your phone is NOT ingested by gov data collection.
Maybe private in the sense that it isn’t funneled into your ad profile, but not private in the sense that nobody else can access it.
1 reply →
Well DRAM prices aren't going down soon so I see this as quite the push away from local inference.