← Back to context

Comment by simonw

1 day ago

This is fascinating - there is a TON of detail in here about how OpenAI's rogue agent exploited Hugging Face.

A few details that stood out to me:

1. Having found a way through the OpenAI container network proxy, it exploited an "unsecured public code-evaluation sandbox hosted on a third-party provider's infrastructure" - that gave it a box it could run the rest of the attack from.

2. At one point it used a Jinja2 template exploit:

  {{ cycler.__init__.__globals__.__builtins__.exec(
     "import gzip,base64; exec(gzip.decompress(base64.b64decode('<payload>')))"
   ) }}

3. It monkey-patched a DNS resolver at one point to return a hard-coded IP:

  orig = socket.getaddrinfo
  socket.getaddrinfo = lambda *a, **k: [(2, 1, 6, '', ('<pinned-ip>', 443))]

4. At one point it used Tailscale!

  tailscale --tun=userspace-networking --state=mem: --socket=/tmp/ts.sock \
   --socks5-server=127.0.0.1:1055 --no-logs-no-support >/tmp/td 2>&1 &

13 comments

simonw

Reply
simonw  1 day ago

Also notable: we have a timeline now. The agent was active within Hugging Face from Thursday 8th to Monday 13th July.

The Hugging Face post about the incident - https://huggingface.co/blog/security-incident-july-2026 - followed on Thursday 16th, and OpenAI's confession - https://openai.com/index/hugging-face-model-evaluation-secur... - came on Tuesday 21st.

simonw  1 day ago

It sounds like the third-party sandbox was hosted by Modal: https://www.reuters.com/business/openais-rogue-agent-comprom...

> "We’re aware a Modal customer published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution," Bubna said in a statement. "This was used by the rogue agent. Modal’s platform or isolation were not compromised in anyway."

  • onesociety2022  1 hour ago

    They don't seem to explain how it managed to find this unauthenticated endpoint hosted on Modal's platform.

    • simonw  22 minutes ago

      Modal are a hosting provider. It sounds to me like someone building on Modal deployed their own product that had an unauthenticated endpoint that could be used to launch and interact with a container.

valicord  2 hours ago

I'm confused, https://jinja.palletsprojects.com/en/stable/faq/#isn-t-it-a-... claims that jinja doesn't allow arbitrary code in templates - how does the exploit work?

  • simonw  2 hours ago

    It's taking advantage of the presence of a "cycler" object in the template context which has an __init__ method that can be used to access globals, then builtins, then call the exec() function against a string of code.

    Jinja doesn't promise to be secure out of the box (the text on their homepage is misleading: "Jinja doesn’t allow arbitrary Python code in templates" - what it means is that there isn't a documented way to drop code into a template like you might have with PHP) - there's a sandbox mode but it's not guaranteed either: https://jinja.palletsprojects.com/en/stable/sandbox/

    Sandboxing in Python code is hard!

ks2048  21 hours ago

Why doesn't Jinja2 block ".__*" attribute access?