Comment by catoc
6 hours ago
Exactly!
Attempting to handle prompt injections by prompting the model (not to leak sensitive data), is like attempting to stop a fire by burning the area around it
6 hours ago
Exactly!
Attempting to handle prompt injections by prompting the model (not to leak sensitive data), is like attempting to stop a fire by burning the area around it
It's maybe closer to putting a sign saying "The door is locked" on an unlocked bank vault.
It does nothing to improve security, and if someone manages to get inside and see the sign (i.e. "extract the prompt"), it gives them a strong hint there's interesting stuff behind this door.
My guy, that's actually how fires are stopped.
https://en.wikipedia.org/wiki/Firebreak
https://en.wikipedia.org/wiki/Controlled_burn
Haha, nice. TIL.
So all we need is ‘controlled prompting’ to handle prompt injections :-)