Comment by SilasX

Right, for the OpenAI case to be analogous, they would have to switch to a system where your chats are homomorphically encrypted -- i.e. OpenAI does all its operations without knowing either the input or output plaintext. In that case, they'd only have encrypted chats to begin with, and would have to somehow get your key to comply with a warrant for the plaintext.

And note: the above scenario is not likely anywhere in the near future, because homomorphic encryption has something like a million times overhead, and requires you to hit the entire database on every request, when state-of-the-art LLM systems are already pushing the limits of computation.