But, it's still running on my desktop/laptop. I don't trust them to run on my machine. But, I guess I could run one VM with a desktop to contain the desktop app. Or, just keep using CLI agents.
For local tasks you can only give agents delegated that execute your deterministic read or write on an allowed set of files(e.g pi does this) and execute rights only on containers with no network access. That should get you 95% unblocked for most tasks you want to do with an LLM pretty safely.
You can do a brainstorming with web on a remote container prototyping based on that brainstorm on another container with no network access.
The one thing that is less trustworthy is using local agents for service management, you definitely want to have them scoped to dev/testing. I would never trust an agent to execute any command in production or sensitive data at all
Is the trust concern for the agent running in any form on your machine? Like in a VM on your machine as well or do you mean on the host itself?
I have read about people giving an agent full access to their main system saying they have nothing of value. To me, that's a strange opinion to have with the distinction between what's private and what's secret.
I don't run agents directly on my desktop/laptop machine. I run them in VMs or containers (sometimes in containers on VMs). There have been too many credentials stealing exploits via prompt injection and the like for me to be willing to let an agent roam around on my personal system.
I've also started creating new github deploy keys for each repo in use on a VM, so the blast area for any given agent disaster is "a couple/few github repos and whatever credentials were needed for the agent/model".
I wouldn't let a coworker, even one I know pretty well, log into my personal account on my machines...why would I let an agent that can be tricked into uploading all my credentials to an attackers web server?
The agents have sandboxes, but those are loose. Not enforced by anything outside of the agent harness itself.
I mean, if the execution happen on the VM then the problem is trust on the programs and then you can't trust any program by that logic? That or you think AI-companies software is serious slop.
I do, in fact, think the AI companies software is serious slop. I've read some of the Claude Code source.
And, we're not talking about hypothetical attacks here. Prompt injection attacks have happened. Supply chain attacks that agents fell for have happened.
I don't run anything but the agent and the project it's working on and the tools it needs to work on the project in the VM.
You can't see how the agent having no access to anything other than what it's working on is safer than the agent having access to my home directory with all of my credentials?
Look, you do whatever you want to do with your agents and your computer. I'm going to...contain them.
Seriously, you dont see any difference? A agent is non deterministic and may delete or change you data as a normal matter of operations. A browser, barring bugs or security issues, would not delete or modify the data you have outside the browser.
That's also true if you're running the agent directly on your laptop OS.
In that case, maybe you want VMs at hosting providers. There are companies building ephemeral VM and container orchestration layers for this kind of thing, I haven't played with them, though. It seems like a reasonable idea, though. One isolated environment per project or repo. Only the secrets needed for that one project and an agent that can't reach outside of it.
I've considered building something along those lines, and actually do run my security auditing benchmarks in containers automatically (that was originally to prevent the models from cheating, because you can disable network, but it has other pleasant side effects).
It's actually not that big of a lift these days to spin up containers on-demand and put just what's needed inside it (including the authentication info for the agent). I probably should automate it..right now I just have four permanent VMs setup for my various types of work: My day job, my open source projects, my benchmark and security work, and some side projects. Plus some temporary ones for experiments.
No, it actually continues running headless on the host, and you can reconnect from another laptop or mobile phone, or even ssh to the host and attach to the session. At least Codex desktop app works this way.
But, it's still running on my desktop/laptop. I don't trust them to run on my machine. But, I guess I could run one VM with a desktop to contain the desktop app. Or, just keep using CLI agents.
For local tasks you can only give agents delegated that execute your deterministic read or write on an allowed set of files(e.g pi does this) and execute rights only on containers with no network access. That should get you 95% unblocked for most tasks you want to do with an LLM pretty safely.
You can do a brainstorming with web on a remote container prototyping based on that brainstorm on another container with no network access.
The one thing that is less trustworthy is using local agents for service management, you definitely want to have them scoped to dev/testing. I would never trust an agent to execute any command in production or sensitive data at all
Is the trust concern for the agent running in any form on your machine? Like in a VM on your machine as well or do you mean on the host itself?
I have read about people giving an agent full access to their main system saying they have nothing of value. To me, that's a strange opinion to have with the distinction between what's private and what's secret.
I don't run agents directly on my desktop/laptop machine. I run them in VMs or containers (sometimes in containers on VMs). There have been too many credentials stealing exploits via prompt injection and the like for me to be willing to let an agent roam around on my personal system.
I've also started creating new github deploy keys for each repo in use on a VM, so the blast area for any given agent disaster is "a couple/few github repos and whatever credentials were needed for the agent/model".
I wouldn't let a coworker, even one I know pretty well, log into my personal account on my machines...why would I let an agent that can be tricked into uploading all my credentials to an attackers web server?
The agents have sandboxes, but those are loose. Not enforced by anything outside of the agent harness itself.
15 replies →
I mean, if the execution happen on the VM then the problem is trust on the programs and then you can't trust any program by that logic? That or you think AI-companies software is serious slop.
I do, in fact, think the AI companies software is serious slop. I've read some of the Claude Code source.
And, we're not talking about hypothetical attacks here. Prompt injection attacks have happened. Supply chain attacks that agents fell for have happened.
https://venturebeat.com/security/six-exploits-broke-ai-codin...
I'm going to "security in depth" these gullible little thinky guys in my computer, but you do what makes you happy.
Slop is less of a problem than the incentive such companies have to “accidentally” hoover up whatever data is accessible.
Do you also run your browser in the VM? Why would an agent be less trusted than any other piece of software?
I don't run anything but the agent and the project it's working on and the tools it needs to work on the project in the VM.
You can't see how the agent having no access to anything other than what it's working on is safer than the agent having access to my home directory with all of my credentials?
Look, you do whatever you want to do with your agents and your computer. I'm going to...contain them.
https://venturebeat.com/security/six-exploits-broke-ai-codin...
Seriously, you dont see any difference? A agent is non deterministic and may delete or change you data as a normal matter of operations. A browser, barring bugs or security issues, would not delete or modify the data you have outside the browser.
But then I close my laptop and it’s not running on the headless host anymore right
That's also true if you're running the agent directly on your laptop OS.
In that case, maybe you want VMs at hosting providers. There are companies building ephemeral VM and container orchestration layers for this kind of thing, I haven't played with them, though. It seems like a reasonable idea, though. One isolated environment per project or repo. Only the secrets needed for that one project and an agent that can't reach outside of it.
I've considered building something along those lines, and actually do run my security auditing benchmarks in containers automatically (that was originally to prevent the models from cheating, because you can disable network, but it has other pleasant side effects).
It's actually not that big of a lift these days to spin up containers on-demand and put just what's needed inside it (including the authentication info for the agent). I probably should automate it..right now I just have four permanent VMs setup for my various types of work: My day job, my open source projects, my benchmark and security work, and some side projects. Plus some temporary ones for experiments.
No, it actually continues running headless on the host, and you can reconnect from another laptop or mobile phone, or even ssh to the host and attach to the session. At least Codex desktop app works this way.
Codex, Claude Code, ZAI — they continue work in headless mode, when you close your laptop, if you have connected to remote machine
Examples here?
[dead]
What's stopping a CLI from doing the same?
I've never used IDEs and never will, why are these things being constantly shoved down our throats?