Comment by ipython
1 month ago
The research is interesting but I cringe every time there is a reference to “authorization” or that the roles form the “security architecture” of an llm.
LLMs in their current form provide no security boundaries or guarantees full stop. We need to be clear about this otherwise we end up with truly insecure architectures that can be fooled with the 2026 equivalent of a cereal box whistle.
100%. Anyone who is feeding unsanitized input to an LLM is doing it wrong. It'd be just like letting users craft their own SQL queries. I think the security aspect raises an interesting (if awkward) question:
How do you sanitize inputs to an LLM? Like how can you even make a secure user-facing product with this thing?
Maybe I'm lacking imagination, but it seems to me all the great "natural language interface" solutions this is supposed to enable are pretty badly hobbled by this issue.
> It'd be just like letting users craft their own SQL queries.
The original purpose of SQL was end users entering queries. Configure it right and the database server is perfectly capable of constraining what arbitrary SQL input can do.
LLMs are not like that. The input literally cannot be sanitized. You have to manage the boundaries outside that scope.
https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
https://simonwillison.net/2025/Apr/11/camel/
Even your discussion makes it "sanitized input" simply doesn't exist in relation to an LLM. At best it seems like one can prefix and filter input as much as possible, monitor the results but never assume that you are done.
If that's the case then user-facing products that can take any useful action are strictly off the table.
13 replies →