Comment by throwaway314155
14 hours ago
Your emphasis here seems to downplay the end-result of the attack - which is arbitrary code execution from a seemingly innocent URL merely being read by the LLM. The ACE is pulled off without the user knowing, and seemingly without agent or its auto-mode classifier knowing. There are at the very least _elements_ of prompt injection/jailbreaking in here. The LLM reads content and performs actions described failing to stop itself.
Well, the point is it's more than being read: the task involves downloading and interpreting information which may also involve code. I think it is a pretty good demonstration of what filtering at the LLM-interaction boundary can and can't protect against. And also a good demonstration of how agents will take more action than you might naively assume when given a task unless you specifically limit them.
The end result is the same but it's more of a mismatch of expectations from the user and plain old trickery than it is an attack managing to misdirect the goals of the agent, and it's worth being clear about where the issue is and isn't.