← Back to context

Comment by TheTaytay

6 days ago

I keep getting hung up on securely storing and using secrets with CLI vs MCP. With MCP, you can run the server before you run the agent, so the agent never even has the keys in its environment. That way. If the agent decides to install the wrong npm package that auto dumps every secret it can find, you are less likely to have it sitting around. I haven’t figured out a good way to guarantee that with CLIs.

14 comments

TheTaytay

Reply
Aperocky  6 days ago

A CLI can just be a RPC call to a daemon, exact same pattern apply. In fact my most important CLI based skill are like this.. a CLI by itself is limited in usefulness.

  • TheTaytay  6 days ago

    That was the same conclusion I reached! However, this also gave me some evidence that maybe I wanted MCP? I realized that my pattern was going to be:

    Step 1) run a small daemon that exposes a known protocol over a unix socket (http, json-rpc, whatever you want), over a unix socket. When I run the daemon, IT is the only that that has the secrets. Cool! Step 2) Have the agent run CLI that knows to speak that protocol behind the scenes, and knows how to find the socket, and that exposes the capabilities via standard CLI conventions.

    It seems like one of the current "standards" for unix socket setups like this is to use HTTP as the protocol. That makes sense. It's ubiquitous, easy to write servers for, easy to write clients for, etc. That's how docker works (for whatever it's worth). So you've solved your problem! Your CLI can be called directly without any risk of secret exposure. You can point your agent at the CLI, and the CLI's "--help" will tell the agent exactly how to use it.

    But then I wondered if I would have been better off making my "daemon" an MCP server, because it's a self-describing http server that the agent already knows how to talk to and discover.

    In this case, the biggest thing that was gained by the CLI was the ability of the coding agent to pipe results from the MCP directly to files to keep them out of its context. That's one thing that the CLI makes more obvious and easy to implement: Data manipulation without context cluttering.

JamesSwift  1 day ago

Just wrap it with a script that handles the auth for you and the AI doesnt realize auth is even needed. I put my creds in ~/.config/ and write bash wrappers that read those and proxy the values into the api calls as needed.

usrbinbash  6 days ago

And in a skill, I can store the secret in the skill itself, or a secure storage the skill accesses, and the agent never gets to see the secret.

Sure, if I want my agents to use naked curl on the CLI, they need to know secrets. But that's not how I build my tools.

  • lukewarm707  6 days ago

    what stops the agent from echoing the secure storage?

    what i see is that you give it a pass manager, it thinks, "oh, this doesn't work. let me read the password" and of course it sends it off to openai.

    • usrbinbash  6 days ago

      > what stops the agent from echoing the secure storage?

      The fact that it doesn't see it and cannot access it.

      Here is how this works, highly simplified:

          def tool_for_privileged_stuff(context:comesfromagent):
        creds = _access_secret_storage(framework.config.storagelocation)
        response = do_privileged_stuff(context.whatagentneeds, creds)
        return response # the agent will get this, which is a string

      This, in a much more complex form, runs in my framework. The agent gets told that this tool exists. It gets told that it can do privileged work for it. It gets told how `context` needs to be shaped. (when I say "it gets told", I mean the tool describes itself to the agent, I don't have to write this manually ofc.)

      The agent never accesses the secrets storage. The tool does. The tool then uses the secret to do whataever privileged work needs doing. The secret never leaves the tool, and is never communicated back to the agent. The agent also doesn't need, or indeed can give the tool a secret to use.

      And the "privileged work" the tool CAN invoke, does not include talking to the secrets storage on behalf of the agent.

      All the info, and indeed the ability to talk to the secrets storage, belongs to the framework the tool runs in. The agent cannot access it.

      2 replies →