Comment by simonw
3 days ago
These aren't vulnerabilities in LLMs. They are vulnerabilities in software that we build on top of LLMs.
It's important we understand them so we can either build software that doesn't expose this kind of vulnerability or, if we build it anyway, we can make the users of that software aware of the risks so they can act accordingly.
Right; the point is that it's the software that gives "access to sensitive data, user input and the ability to make arbitrary http calls" to the LLM.
People don't think of this as a risk when they're building the software, either because they just don't think about security at all, or because they mentally model the LLM as unerringly subservient to the user — as if we'd magically solved the entire class of philosophical problems Asimov pointed out decades ago without even trying.