Comment by roughly
2 days ago
Looks like Confer is hosting its own inference: https://confer.to/blog/2026/01/private-inference/
> LLMs are fundamentally stateless—input in, output out—which makes them ideal for this environment. For Confer, we run inference inside a confidential VM. Your prompts are encrypted from your device directly into the TEE using Noise Pipes, processed there, and responses are encrypted back. The host never sees plaintext.
I don’t know what model they’re using, but it looks like everything should be staying on their servers, not going back to, eg, OpenAI or Anthropic.
That is a highly misleading statement: the GPU runs with real weights and real unencrypted user plaintext, since it has to multiply matrices of plain text, which is passed on to the supposedly "secure VM" (protected by Intel/Nvidia promises) and encrypted there. In no way is it e2e, unless you count the GPU as the "end".
It is true that nVidia GPU-CC TEE is not secure against decapsulation attacks, but there is a lot of effort to minimize the attack surface. This recent paper gives a pretty good overview of the security architecture: https://arxiv.org/pdf/2507.02770
So what you are saying is that all the TEE and remote attestation and everything might work for CPU based workflows but they just don't work with GPU effectively being unencrpyted and anyone can read it from there?
Edit: https://news.ycombinator.com/item?id=46600839 this comment says that the gpu have such capabilities as well, So I am interested what you were mentioning in the first place?
> Looks like Confer is hosting its own inference
Even so, you're still exposing your data to Confer, and so you have to trust them that they'll behave as you want. That's a security problem that Confer doesn't help with.
I'm not saying Confer isn't useful, though. e2ee is very useful. But it isn't enough to make me feel comfortable.
> you're still exposing your data to Confer
They use a https://en.wikipedia.org/wiki/Trusted_execution_environment and iiuc claim that your client can confirm (attest) that the code they run doesn't leak your data, see https://confer.to/blog/2026/01/private-inference/
So you should be able to run https://github.com/conferlabs/confer-image yourself and get a hash of that and then confer.to will send you that same hash, but now it's been signed by Intel I guess? to tell you that yes not only did confer.to send you that hash, but that hash is indeed a hash of what's running inside the Trusted Execution Environment.
I feel like this needs diagrams.
> I feel like this needs diagrams.
And there's the problem.
All of that stuff is well and good, but it seems like I have to have a fair degree of knowledge and technical skill, not to mention time and effort, to confirm that everything is as they're representing. And it's time and effort I'd have to expend on an ongoing basis.
That's not an expectation I could realistically meet, so in practice, I still have to just trust them.
2 replies →
As I read it, the attestation is simply that the server is running a particular kernel and application in the Secure Enclave using the hardware’s certification. That does not attest that there is no sidechannel. If exfiltration from the TEE is achieved, the attestation will not change.
To put it another way, I am quite sure that a sufficiently skilled (or privileged: how do you know the manufacturer is not keeping copies of these hardware keys?) team could sit down with one of these enclave modules and figure out how to get the memory image (or whatever) out without altering the attested signature.
2 replies →
That’s true, but it’s still a distinct threat model from “we use the API of a company run by one of the least trustworthy humans on the planet.” We can talk through side channel attacks and whatnot, but we’re discussing issues with Confer’s implementation, not trusting a different third party.
We'll add that link to the toptext as well. Thanks!
(It got submitted a few times but did not get any comments - might as well consolidate these threads)