Comment by jostmey

I disagree. I think targeting running models on end user devices is a good goal, and it's the ideal case for user privacy and latency.

The human brain consumes around 20 watts, while of course there are substantial differences with implementation I think it's reasonable to draw a line and say that eventually we should expect models to hit similar levels of performance per watt. We see some evidence now that small models can achieve high levels of performance with better training techniques, and it's perfectly conceivable that acceptable levels of performance for general use will eventually be baked into models small enough to run on end hardware. And at the speed of development here, "eventually" could mean 1-2 years.

Actually, it's more of a sad capitulation to lazy armchair "analysts" and "pundits" who whined incessantly that Apple was "behind on AI," without taking stock of the fact that Apple does not NEED "AI." It does not serve their core businesses, product line, or users.

Instead of loudly jumping on this depressing bandwagon, Apple should have quietly improved Siri and then announced it when it was WORKING.

Shoulda kept Scott Forstall

> The very fact that Apple thought they were going to run AI on iPhones

Nope

https://security.apple.com/blog/private-cloud-compute/