Comment by elpakal

1 day ago

They announced this at wwdc this year in “Foundation Models”. Developers have access to an on-device llm (not sure how good it is yet).

Yeah but they're just too small to do anything useful with yet. Like we're in this weird state where you can't easily sell usage based pricing through appstore payments (and customers don't really understand usage anyways). So you need to sell access to an agent via a subscription, but your costs are 90% usage based so it's hard to price. If appstore developers could use a quota of access tokens from a users apple intelligence subscription we could offer AI agents for $3-5/mo and they would be actually usable! But if you need to pay for inference costs it has to be $10-20/mo. It's just a lame experience and makes the web the place to build agents even though they'd be more useful on mobile devices.