Comment by jameslk
1 day ago
It's not a perfect security model. Between the friction and all caps instructions the model sees, it's a balance between risk and simplicity, or maybe risk and sanity. There's ways I can imagine the concept can be hardened, e.g. with a server layer in between that checks for things like dangerous actions or enforces rate limiting
If all you're doing is telling an LLM to do something in all caps and hoping it follows your instructions then it's not a "security model" at all. What a bizarre thing to rely on. It's like people have literally forgotten how to program.
These people often never knew in the first place.
“AI changes everything!”
Thank you for saying this. I read this and was like: wtf?
Love agents, but the security risk is insane.
If I were the CEO of a place like Plaid, I'd be working night and day expanding my offerings to include a safe, policy-driven API layer between the client and financial services.
What if instead of allowing the agent to act directly, it writes a simple high-level recipe or script that you can accept (and run) or reject? It should be very high level and declarative, but with the ability to drill down on each of the steps to see what's going on under the covers?