Yeah exactly. The MacBook Pro is by far the most capable consumer device for local LLM.
A beefed up NPU could provide a big edge here.
More speculatively, Apple is also one of the few companies positioned to market an ASIC for a specific transformer architecture which they could use for their Siri replacement.
(Google has on-device inference too but their business model depends on them not being privacy-focused and their GTM with Android precludes the tight coordination between OS and hardware that would be required to push SOTA models into hardware. )
I see. It'll be interesting to see how much on-device models take off for consumers, when off-device models will be so much more capable. In the past, the average consumer has typically been happy to trade privacy for better products, but maybe it will be different for llms.
This is only true if you consider AI to be LLMs and chatbots. The non-LLM AI built into just the iPhone camera is almost certainly the largest scale consumer deployment of any AI but largely goes unnoticed because it works so well.
Yeah exactly. The MacBook Pro is by far the most capable consumer device for local LLM.
A beefed up NPU could provide a big edge here.
More speculatively, Apple is also one of the few companies positioned to market an ASIC for a specific transformer architecture which they could use for their Siri replacement.
(Google has on-device inference too but their business model depends on them not being privacy-focused and their GTM with Android precludes the tight coordination between OS and hardware that would be required to push SOTA models into hardware. )
I see. It'll be interesting to see how much on-device models take off for consumers, when off-device models will be so much more capable. In the past, the average consumer has typically been happy to trade privacy for better products, but maybe it will be different for llms.
They are well positioned but have a history of screwing up their AI plays, I hope they can get it right.
This is only true if you consider AI to be LLMs and chatbots. The non-LLM AI built into just the iPhone camera is almost certainly the largest scale consumer deployment of any AI but largely goes unnoticed because it works so well.