Comment by trollbridge
4 hours ago
A good deal of the power of agents is that they simply reduce friction and figure out how how to solve cumbersome but obviously possible tasks. That often means workarounds for security.
The more security conscious they are, the less useful they are.
One can imagine an LLM paired with a bit-colour system that never permits red data to be used in green contexts. Complex tasks could be completed only if they didn't violate security restrictions.
But we already have that, and the security system doesn't work.