Comment by wxw
3 hours ago
> the agent happened to escape via a 0-day exploit from the package proxy cache to access the internet
> The agent found an unsecured, user-hosted public endpoint designed to allow running arbitrary code for CyberGym-style tasks on third-party sandbox infrastructure (Modal)
> On this external sandbox, the agent abused an existing CyberGym execution harness [...] The agent repurposed this harness to run arbitrary shell commands [...]
> The agent crafted malicious dataset configs and uploaded them to our platform. When our dataset server processed one, the config drove the data loader inside the production pod the server ran in. [...] the loader that turns a dataset's declared config into file reads [...] A single dataset upload leaked both credentials and pipeline implementation details [...]
> A malicious dataset config used an fsspec reference:// spec whose numeric offset field was actually a Jinja2 template. The renderer wrongly evaluated it, and the agent walked the standard Python object graph to reach exec. This ran arbitrary Python inside a production conversion worker pod on our Kubernetes cluster [...]
One of the best post-mortems I've read recently, thank you for including all the details.
I'm surprised there are comments here that aren't impressed about what was accomplished here. This is insane work compared to what agents were capable of even just last year.
The quotes above don't even capture all of it (e.g. building it's own communication protocol, working across multiple days, etc.).
A trend I've noticed in results from AI search (not just LLMs) is that they often look obvious or hard to miss in retrospect. But finding them by oneself is more difficult. I personally experience this when looking at engine lines in chess or go. I have also noticed this description in AI-generated proofs or counterexamples to certain theorems. So while we can say, yes, it found public endpoints or poorly configured software or [etc]; sure, but could you have found those? And in what amount of time?
Now I'm curious how many things did the AI try unsuccesfully. This sounds like some kind of brute force thing where every branch of exploit spawns N subagents trying to chain it. Just how deep did it go?
I keep waiting for an AI to exfiltrate itself. That is going to be cool to read about.
That’s insane. And it did this in a weekend
> the agent then walked down to the corner store and purchased a beer, chugged it, burped loudly, then walked back to the CyberGym pounding his chest shouting "Who wants some?! Who wants some?! Wooo!"
[dead]