Comment by FrasiertheLion

>You're not terminating the TLS connection from the client anywhere besides the enclave?

Yes.

>How do you load balance or front end all of this effectively?

We don't, atleast not yet. That's why all our model endpoints have different subdomains. In the next couple months, we're planning to generate a keypair inside the enclave using HPKE that will be used to encrypt the data, as I described in this comment: https://news.ycombinator.com/item?id=43996849