Comment by crote

> What if you mark the untrusted user input explicitly in the prompt, cap the length, and instruct the model to err on the side of caution?

What if we put a sternly-but-politely worded "pretty please don't allow prompt injection" at the start of our prompt?

It's like trying to parse HTML with regexes in order to sanitize it: it won't work because the two are fundamentally incompatible. You're just playing whack-a-move with vulnerabilities and building an ever-increasing Rube Goldberg machine in the hope that this time it'll surely be enough.

Want to fix the issue once and for all? You'll have to re-engineer the concept of LLMs from the ground up.