Comment by jdzikowski
4 days ago
I think there might be also another possible way to handle the sensitive data issue. Maybe in the future instead of putting agent into the cloud sandboxes, we let agents work locally and put sensitive data into "cages" or "vaults" agents can't access.
Both Apple and Google are building the infrastructure for on device agents. I'm not as familiar with Apple's approach, but I've been hands-on with android AppFunctions. If you're familiar with Android ContentProviders and bound services, you've seen how apps can be custodians of the data they acquire and use.
AppFunctions enable tool calling with descriptions that are legible to LLM based This puts the apps in control of what agents can access, which is something they already mostly do.
And you won't be able to disable it.
You can certainly avoid using agents. While I use coding agents and LLMs to research certain topics, I don't have a lot of use for other ways AI has been integrated into mobile devices.
When I first experimented with AppFunctiions, I was impressed with what the very limited test agent could do combining information from multiple apps in useful ways.
I'm working on a product for elder care. I could've spent quite a bit of time integrating Google maps and creating a bespoke automation that would tell care providers about shopping for specific items on their way to a care recipient home. Instead I just prompted the agent to plan that trip. Even at an early prototype stage, using a test agent that had limited access to apps, the result felt magical: agents automate automations. You don't stop to sharpen the saw, you just ask for the right saw for the job and there it is.
Unlike early attempts at agents that need sketchy access to grab screenshots and use your passwords, app tool calling feels a lot safer and more orderly.
[dead]
>data into "cages" or "vaults" agents can't access.
This is likely worse as it turns people into the kind of "3 laws safe" thinking. Reality has shown us there is no such thing as an unbreakable cage (well, maybe a black hole is, but you can't get anything back out of it in a useful time frame).
By the time you realize the cage was flawed your data may already be distributed far and wide.
I agree. But isn't it somewhat similar to syncing data with iCloud/OneDrive/GoogleDrive? A convenience service that manages your sensitive data. Just replace sync with cage. It's leakable, but might be also more convenient and cheaper than agents in the cloud.
[flagged]