Comment by skinfaxi
10 hours ago
This is probably a dumb question but why can't we use message signing to segregate the streams or at least add repudiation? The message telling you to look at the doc is signed but the doc is not signed and thus not treated like chat input.
The way the LLM "looks" at a document is it receives a single blob of data containing both instructions and the doc itself.
There aren't two different inputs to an LLM, just one. You can put any text you want in the input (the "context window"), but fundamentally the current architecture of LLMs is a function taking one argument. So signing doesn't help because the portion that's signed is commingled with the portion that's not signed, and sometimes the LLM will "do things" based on the unsigned portions' contents.