Comment by nerder92

Thanks for chiming in.

I agree this is not a one-click account takeover.

But I think point 2 is broader than that. The user does not need to ask about the malicious transaction specifically. Any normal question that makes the agent fetch recent transactions could bring the attacker-controlled text into the LLM context.

Unless I missed it they didn't provide any proof of this actually working. Really seems like a thing veiled advert for their product

This is similar to scam where people are sent messages about bad transaction with a fake link to the bank to verify it. Some attackers have gotten Paypal to send notifications that have the link. People are supposed to check the source and go directly to bank, and this will bypass that.

Depending on how much access the AI agent has, there are worse things to inject it with than a link.

People already click suspicious emails that ask them to login. At a high number of attempts, some chickens will be caught. However, people are now weary of emails since there is a lot of phishing there. On the other hand, the AI assistant env. could be considered "safe" by users because it's stuff coming from the bank. So they are more likely to fall for it. (honestly, unless you are a dev and aware of prompt injection, I don't see why the users wouldn't fall for it).

I think the critical part is that it launders an arbitrary URL as trustworthy. The alternative is “Don’t trust anything our bot says at face value, please.”

I think a better criticism is allowing arbitrary text (including URLs) in a transaction description.