Comment by alwillis
6 days ago
> Private cloud is used AFIAK for virtually 0 use cases so far.
Applications using Apple's foundation models can seamlessly switch from on-device models to Private Compute Cloud.
Research is already showing the use of LLMs for people's most intimate relationship and medical issues. The usual suspects will try to monetize that, which why Private Cloud Compute is a thing from the jump.
> Then they've got OpenAI/Gemini/Anthropic via API. But this completely goes against all their private cloud messaging
Using ChatGPT via Siri today, no personally identifying information is shared with OpenAI and those prompts aren't used for training. I suspect Apple would want something similar for Google, Anthropic, etc.
At some point, there will be the inevitable enshitification of AI platforms to recoup the billions VCs have invested, which means ads, which won't happen to Apple users using foundation model-based apps.
> Nearly all their devices do not have enough RAM and
Every Apple Silicon Mac (going back to the M1 in 2020) can run Apple Intelligence. 8 GB RAM is all they need. Every iPhone 15 Pro, Pro Max and the entire 16 line can all run Apple Intelligence.
Flagship iPhone 17 models are expected to come with 12 GB of RAM and all current Mac models come with at least 16 GB.
Apple sells over 200 million iPhones in a given year.
There's no doubt Apple stumbled out of the gate regarding AI; these are early days. They can't be counted out.
8GB RAM is not enough for a semi-decent model IMO. 12/16GB is better (4GB for model and 8GB for OS) and really if you were going hard on device you'd probably want more like 32GB (24GB for model + 8GB for everything else - you'd be able to run a 13b param model with larger context size with that).
Even still though people are used to the quality of huge frontier models, so it will feel like a massive downgrade on many tasks. The _big_ problem with all this is chained tool calling. It uses context SO quickly and context needs a lot of (V)RAM. This also completely undermines the privacy argument you make, because it will need to ask personal data if using OpenAI and put it in the prompt.
Yes I noticed Apple shipping higher RAM but it will take years for this to feed through to a sizeable userbase, and people are quickly getting ingrained to use an app like ChatGPT instead of OS level features. Even more so given what a flop Apple Intelligence 1.0 has been.
The key problem they've got is they've went hard on privacy (which means it is hard to square that with going all in on 3rd party APIs) but they've also been incredibly stingy with RAM historically, which really nerfs their on device options. Private compute is an interesting middle ground but their model options are incredibly limited currently.
> 8GB RAM is not enough for a semi-decent model IMO.
Apple's ~3 billion parameter on-device model is about as good as it gets on a smartphone, especially for the functions it was designed for: writing and refining text, prioritizing and summarizing notifications, creating images for conversations, and taking in-app actions.
Every Mac comes with at least 16 GB of RAM; while every iPhone comes with 8 GB of RAM, some models of the iPhone 17 will have 12 GB.
Remember, an app using the on-device model can seamlessly shift to a much bigger model via Private Cloud Compute without the user having to do anything.
If the user enables it, Apple's Foundation Model can use ChatGPT in a privacy preserving way. By the fall, Gemini and Sonnet/Opus could be options as well.
Again, ChatGPT is used in a privacy preserving way; you don't need an account: "Use ChatGPT with Apple Intelligence on iPhone" [1].
[1]: https://support.apple.com/guide/iphone/use-chatgpt-with-appl...