Comment by owebmaster
2 days ago
> The agent should be treated as an untrusted user in your client,
An untrusted user in a client is a hacker/invasor, not an agent.
2 days ago
> The agent should be treated as an untrusted user in your client,
An untrusted user in a client is a hacker/invasor, not an agent.
That’s not really a reason not to treat the agent like it’s “rogue”. The point is, if it accepts any untrusted inputs then, from a security perspective, it is possible for any given (untrusted) input to contain a prompt injection payload that jailbreaks the model and tells it to do things it shouldn’t do.
As such, it can be told to do bad stuff in a way that can’t be prevented and therefore should not be given read access to anything you don’t want others to know about, nor write access to any data of which you care about the integrity.
That is out of scope of the service. What kind of user agent the actual user deputizes to interact with a service, is the user's own choice and responsibility. In general, it's not something a service can solve on their end.
Services can certainly make this safer by providing means to get more restricted credentials, so that users can deputize semi-trusted delegates, such as agents vulnerable to injection.
The important point being made in this discussion is that this is already a common thing with OAuth, but mostly unheard of with web sessions and cookies.
an untrusted, but permitted, user is why sandboxes exist. There are plenty of times you want to allow an untrusted user to have capabilities in a system, that's why you restrict those capabilities.
a sandboxed user is not an untrusted user of the client but an unstrusted user of the host, that is why the client is sandboxed.
sandboxing is a general term for actor isolation, and its context agnostic.
For example, when you use the sandbox attribute on an iframe in a web application, it's not the user that's untrusted, it's some other user that's attempting to trigger actions in your client.
1 reply →