Comment by verdverm
4 days ago
I switched to Pi last week, have not seen anything about self modification, but have seen a lot about security, the minimalism and customization by user are closest in common selling points from exiting users
you cannot have both and be serious about security
to be clear, I'm talking about hot reload, agent can manipulate its own environment in real time, that's asking for trouble, put some controls on that or reap the consequences from loss of control
Unless I’m out of the loop, Pi can self modify in the sense that plugins are just code it can edit. It doesn’t live reload that I’m aware of.
Pi’s whole UI is customizable, so a lot of people were having their agents build their own interfaces.
The distinction is probably irrelevant for most people. I secure the whole thing in a container, because I lack the willpower to audit a bajillion lines of LLM code just inside the harness.
See my peer comment for some other thoughts.
What stops your agent from adding a new MCP server on its own? Would you consider that a gap in security posturing?
Pi notably ships without MCP support (or it did, I can't remember if they're adding it to Pi main or just the durable one). From a vanilla Pi perspective, I'm pretty sure but not positive that it can't. It would have to install an MCP plugin, and you would have to re-launch Pi to load that plugin.
It does have native tools, but they're also in plugins that aren't hot-reloaded. The LLM would have to write the tool, you close Pi and re-open it before it was available.
> Would you consider that a gap in security posturing?
Honestly, I think trying to secure the harness is a lost cause for the same reasons SELinux never really took off. They solve a real problem in a way so painful people would rather just live dangerously.
I actually wrote a thing to implement my security posture https://github.com/SethCurry/abyss TLDR it pretends to be an ACP agent, but runs the agent inside Docker (with optional bind mounts) and proxies the ACP connection into the container.
I'm also tinkering with WASM-based plugins in the ACP proxy to add some additional security and compliance posturing at the protocol layer (ie agnostic to what the underlying agent is). E.g. I slapped together a plugin that will automatically cancel a prompt mid-turn if an OpenRouter API token shows up in a) the prompt, b) agent output/thinking, c) agent tool calls. ([1], [2], [3]). The grand vision is slapping centralized management on that so security and compliance teams get a "firewall" of sorts to prevent leaking secrets, filter bash commands, censor PII, yada, yada.
1. Screenshot of it blocking something in Zed https://abyss.scurry.io/docs/guides/what-is-abyss/#i-am-not-... (I need to add buffering so it doesn't output any of the key)
2. Implementation of ^ https://github.com/SethCurry/abyss/blob/main/plugins/secrets...
3. Plugin architecture docs https://abyss.scurry.io/docs/guides/04-plugins/02-developing...
8 replies →
> you cannot have both and be serious about security
Why not? I think DSH does this pretty well with an agent preset that exposes the tools required for the agent to modify the harness.
It's kind of like how the kernel can expose interfaces to allow userspace to load kernel modules at runtime: you "just" have to control which processes can use them.
Between that, and a couple of extra injected bwrap args to make my SSH keys and LLM API credentials unreadable from the agent's shell by default, I'm pretty comfortable using the harness outside of a VM for most things.
> Why not?
If an agent can change the harness it is running in, in real-time, all bets are off
There is semantic ambiguity here. An agent being able to write plugin code for the harness it runs in is related, but orthogonal, to the real-time or not availability of that plugin to the current session.
When I have my agents assist with plugins, I pay closer attention and either do the testing myself or have them run the altered harness in another instance.
Part of this is because they have borked themselves and it can be super annoying to undo their f-ups. Little qwen did this once, breaking its own connection to vllm.
I guess for me there is a distinction between an agent I am using interactively and walking through step-by-step, vs an agent that is running unattended, doing web search, communicating with subagents etc. I am ok with giving greater privilege to the former. But yes Qwen-chan accidentally committing suicide, or OOMing its own inference machine is a canon event.
1 reply →
> have not seen anything about self modification
It was the big selling point of the harness a couple months ago.
I suspect there is a bit of language inflation going on here.
1. agent != harness, Pi is a harness, agents running under Pi can do things, Pi is not the actor or agent
2. self modification vs Pi supports customization / extension / plugin
3. hot-reload (real time modification) vs restart/reload (agent can be blocked)
but "self modification" sounds cooler, I guess