Comment by IceWreck
3 days ago
At this point why not make the agents use a restricted subset of python, typescript or lua or something.
Bash has been unchanged for decades but its not a very nice language.
I know pydantic has been experimenting with https://github.com/pydantic/monty (restricted python) and I think Cloudflare and co were experimenting with giving typescript to agents.
This is a really interesting idea. I wonder if something like Luau would be a good solution here - it's a typed version of Lua meant for sandboxing (built for Roblox scripting) that has a lot of guardrails on it.
https://luau.org/
Being unchanged for decades means that the training data should provide great results even for the smaller models.
It means there's also plenty of bad examples in the training data to learn the wrong lessons from though.
They use bash in ways a human never would, and it seems very intuitive for them.
If you present most LLM's with a run_python tool it won't realize that it can access a standard Linux userspace with it even if it's explicitly detailed. But spiritually the same tool called run_shell it will use correctly.
Gotta work with what's in the training data I suppose.
There are a lot of shellscripts holding this world together out there.
1 reply →
Agents really do not care at all how "nice" a language is. You only need to be picky with language if a human is going to be working with the code. I get the impression that is not the use case here though
> Agents really do not care at all how "nice" a language is.
People do care.
> You only need to be picky with language if a human is going to be working with the code.
Sooner or later humans will have to work with the code - if only for their own self-preservation.
> I get the impression that is not the use case here though
If that's not the use case, there's no legitimate use case at all.
I might have misinterpreted what the submitted link is offering then. I thought this was some "secure" sandboxing thing where agents can write ephemeral scripts as tool calls to take arbitrary actions. No human will be looking at that. It's not checked into a repo and is not maintained.
One-off throwaway scripts can be written in literally anything. It does not matter.
> Sooner or later humans will have to work with the code
We want that to be true, but it's starting to look like it might not be.
Humans will eventually have to work with the code, generally at 3am with alarms going off and three tiers of bosses yelling at them on the phone
TIL about Monty. A number of people have tried to sandbox [python,] using python and user space; but ultimately they've all concluded that you can't sandbox python with python.
Virtual Machines are a better workload isolation boundary than Containers are a better workload isolation boundary than bubblewrap and a WASM runtime.
eWASM has costed opcodes; https://news.ycombinator.com/item?id=46825026 re: eWASM and costed opcodes for agent efficiency
> How do these userspace policies compare to MAC and DAC implementations like SELinux AVC, AppArmor, Systemd SyscallFilter, and seccomp with containers for example?
> [ containers/bubblewrap#sandboxing , cloudflare/workerd, wasmtime-mte, ]
"Microsandbox: Virtual Machines that feel and perform like containers" https://news.ycombinator.com/item?id=44137501
microsandbox/microsandbox: https://github.com/microsandbox/microsandbox :
> opensource self-hosted sandboxes for ai agents
I'll add that agents (CC/Codex) very often screw up escaping/quoting with their bash scripts and waste tokens figuring out what happened. It's worse when it's a script they save and re use because it's often a code injection vulnerability.
I want them to be better at it, but given how hard it is for me as a human to get it right (which is to say, I get it wrong a lot, especially handling new lines in filenames, or filenames that start with --) I find it hard to fault them too much.
I've had LLMs write some pretty complex powershell on the fly. Still a shell language but a lot nicer.
Ideally something like nushell but they don't know that well
Bash is ubiquitous and is not going away any time soon. Nothing is stopping you from doing the same thing with your favorite language.
just-bash comes with Python installed, so in a way that's what this has done. I've used this for some prototypes with AI tools (via bash-tool), can't really productionise it in our current setup, but it worked very well and was undeniably pretty cool.
I feel like Deno would be perfect for this because it already has a permissions model enforced by the runtime
Codex has a JS REPL built in now. And pydantic have a minimal version of Python called Monty.
At least for me codex seems to write way more python than bash for general purpose stuff
Agreed! Very notable codex behavior to prefer python for scripting purposes.
I keep telling myself to make a good zx skills or agents.md. I really like zx ergonomics & it's output when it shells out is friendly.
Top comments are lua. I respect it, and those look like neat tools. But please, not what I want to look at. It would be interesting to see how Lua fairs for scripting purposes though; I haven't done enough io to know what that would look like. Does it assume some uv wrapper too?
I came across a coding harness using Lua as its control plane yesterday: https://github.com/hsaliak/std_slop/blob/main/docs/lua_integ...
> std::slop is a persistent, SQLite-driven C++ CLI agent. It remembers your work through per-session ledgers, providing long-term recall, structured state management. std::slop features built-in Git integration. It's goal is to be an agent for which the context and its use fully transparent and configurable.
[dead]