Comment by ealexhudson
6 hours ago
A lot of the solutions in the CPU space involve things like memory allocation flags, NX bits, canaries, etc. that fire deterministically. Those things are fundamentally not applicable to LLMs, and without those things modern software would be in a vastly worse place.
You could imagine that there are things to change around LLM architecture that will improve its ability to reject prompt "injection", but I think it's fundamentally true that from an information theory perspective there's no bright line between "instruction" and "input data" possible.
Nondeterminism is a red herring. There is a bright line between instructions and data right now, in virtually every transcript format. That we have not succeeded in training an LLM to respect it to a very high degree doesn't imply it is impossible; that they are nondeterministic doesn't imply it is impossible; only that we won't succeed 100% of the time.
A cosmic ray (or rowhammer attack) could flip an X bit too, there isn't anything truly deterministic under the sun.