← Back to context

Comment by dtkav

2 days ago

fly.io is doing really good work. I've super enjoyed building our product on their platform. I love fly-replay combined with super fast start-up.

I've been thinking a lot about how to run agents (and skills) securely while giving them a lot of powerful capabilities.

I recently used their macaroons library to turn arbitrary API keys (e.g. for stripe's API) into macaroons. I route requests for an upstream host (like stripe) through Envoy as a mitm proxy which injects the real creds after verifying the macaroon.

It is such a powerful pattern. I'm always worried about leaking sensitive keys through prompt injection attacks (or just sending them to anthropic), but in this model you can attenuate the keys (both capabilities & validity window) client side. The Envoy proxy lives inside my flycast network so it can't be accessed externally.

It would be so cool if fly built something like this into sprites.dev (though I can see how it would be spooky to have fly install their own certs for stripe, etc...)

8 comments

dtkav

Reply
tptacek  2 days ago

If you read Ben Toews work on the tokenizer you have a good sense of where I want Sprites to go with key leaks and prompt injection:

https://fly.io/blog/tokenized-tokens/

  • dtkav  2 days ago

    Awesome stuff! Thanks for the reply.

    Tokenizer is an explicit proxy though right?

    My use case is very similar, but I wanted a transparent proxy so I could run unmodified scripts. It is a tricky design decision though.

    I also mount a little fuse filesystem that mints macaroon on read (with a shorter lifetime, probably inspired by y'all but i forget from where).

    I work on realtime collaboration of markdown files (currently in Obsidian), which has become a shared-context substrate for agents, skills, etc.. Our own company workspace has skills that have scoped access to fly, stripe, gmail, etc. We're definitely drinking the file-over-app personal-software-for-teams Kool-Aid, so the problem space for us includes access control and auditing.

    Love your work :)

    • tptacek  1 day ago

      We have enough control over the execution environment in a Sprite (unlike a Fly Machine, where the implied Linux contract we have with our users gets in the way) that we can trivially hide explicit proxies.

      We can also attach Macaroons to Fly Machines and Sprites for configurable ambient privileges, something I've wanted us to expose as a feature for a very long time.

      3 replies →