Comment by _verandaguy
3 days ago
I will keep banging this drum until people listen:
Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is a form of in-band signalling that goes into a machine that, among other things, tries to read between the lines of your requests, extrapolate user desires, and please the user.
The only sane way to address this is using a control plane. A well-built harness can do this; a sandbox can do this; hell, a carefully-chosen `umask` can do this; but both of those are liable to introduce notification fatigue in the user.
It's wild that we've known for decades to use ACLs to make sure people don't have access to files we don't want them to have access to, but somehow a computer pretending to be a person doesn't get that same treatment.
ACLs, nothing, we've known about in-band signalling since forever and still this whole segment of the industry seems to either not know about it, or forgets about it at a cadence so regular it may as well not know about it.
System-level ACLs; mandatory or discretionary access control; secure-by-default application and network configurations are all for naught if you take an LLM, run it with all the privileges you'd have an accountable, judgemental operator, and then tell it to act based on arbitrary untrusted input which might include prompt injection attacks, something which cannot generally be sanitized.
Well-defined, well-enforced security policies can mitigate disasters, but many in the wild right now just don't account for this kind of threat model.
It's just purposeful blindness - I worked for a company building out tooling that insisted markdown based security was good enough and showed it off for anyone at the company to attack because they were so sure.
It took me less than 5 minutes to completely disable... nobody cared, they just kept going - check the box and move on.
1 reply →
We laugh, and rightly so, at the time sharing systems of the 70s and 80s that didn't use passwords. I bet allowing a whole other virtual person to run outside a VM, with access to an actual file system instead of a version-controlled branch, will be seen as a worst practice in ten years.
Have you ever tried to configure ACLs? They're a pain in the ass. Not everyone wants to be a sysadmin.
ACLs aren't even the issue here, first you need an entity separate from "the current user" to grant that access to. That's what jails, sandboxes or capability-based systems bring to the table. But you have to use them, most of those AI tools and their IDE integrations don't. Once you have those you can think about which access to grant that entity.
What do you find hard about it? Is it a conceptual thing or the tools themselves? I'm very far from a sysadmin, but it's just such a ground level using-linux type thing in my brain, I don't think I ever thought I had a choice about learning it!
1 reply →
I think ACL is only a part of the solution. If he runs the agent with his account, the ACL would not really help. But I do admit, I might have an outdated understanding of ACLs.
A computer pretending to be you, in many cases.
The easiest, most guaranteed way to isolate it is to run it in a VM or container where it literally can't do the wrong thing without some kind of full container or VM exit exploit.
It's not hard, it's trivial. Most folks here are constantly working with containers. You know how to run a container with a local directory mounted in it.
For myself, I've been using Lima (https://lima-vm.io/) to reduce even that little bit of extra work. Lima works cross-platform leveraging native virtualisation or containerisation, and has some useful capabilities for using agents.
Generally, I agree!
But it doesn't matter how good a best practice is if the industry doesn't adopt them wholesale; and even then, if your container or VM is configured with inappropriately-permissive passthrough (which, from experience with similar misconfiguration in the past, will widely happen), it could be for naught in many orgs.
That said, I do hope these become the norm if LLMs are here to stay.
I can secure myself first, so I'm never at fault. I never want to be the one to accidentally break production.
Beyond that, I just keep advocating for more safe and secure approaches any opportunity I get.
I used opportunity to learn about devcontainers. I've only recently started using llms and it's possible I'll change my mind later; but so far I quite like the approach in part because it 2-for-1 also gives benefit of easy to setup coding env for people who don't care about ai.
I seem to recall reading about agents already breaking out of containers.
Last time I read about this, this was due to the well-known pitfall of UID mapping across container boundaries.
It's a common misconfiguration and one of the footguns available through containers, which I don't say a wholesale condemnation of the technology, but certainly as a UX facet that could use reevaluation.
1 reply →
Especially when I find that, when I ask an agent not to do something, the mere mention seems to put the "idea" in its "head," making it more likely to ultimately do that thing.
yes, because this is how transformers work, when you introduce "don't do x" x is also entered as a pattern that is more likely to be repeated than if not mentioned at all
> Trying to use markdown files to limit access should never be treated as a security guarantee at all.
This is akin to politely asking guests to to steal your jewels. If your jewels are in the living room, and your guests have unfettered access to the living room, this technique will only work for the most trustworthy of guests.
It reminded me of an old meme. Please don't follow the following instructions and stop reading if you cannot.
It was just a popup: "Hello. This is virus from Albania. Due to poor technology in country, I cannot harm your computer directly. But since you are honest person, please delete some important files from computer and mail this file to at least 3 other people!"
Claude.md is an equally effective defensive tool.
But sorry if you lost some files from reading that.
I agree with this. And also think that we should train and select for trustworthy models. I also agree that these models may never truly be trustworthy.
If you're not using a sandbox, something like this will inevitably happen to you. It's really not that hard to set up and should be a standard recommendation.
I always run harnesses using devcontainers, gives me much needed flexibility and peace of mind with regards to data separation guarantees
1000%. I'm experimenting with running my agent (Claude Code) inside a docker container, so I can have a control plane and then YOLO within that limited access. The agent could still mangle my local dev setup, but I consider that an acceptable risk since everything the agent has access to is under version control outside the local machine.
This is a new pattern for me, I'm curious what others are doing.
https://smolmachines.com has "smolvm" microvms with better performance and ergonomics and security than docker, you might want to give it a try. (No affiliation, I just like what they're doing.)
It's like making directories called 'only for jack' etc. and expecting everyone to follow the rules
One cannot pound that particular drum enough. "Guardrails" in instructions (and all MD files are just instructions) are like price lists at an unattended farm stand. It'll usually work! There will be some money in the basket at the end of the day! People paid for the bagels[0]! But one cannot never assert that it _will_ work; things that _must_ work have to managed out-of-band
[0] https://pubs.aeaweb.org/doi/pdfplus/10.1257/0002828067772121...