Comment by jiggawatts

The concept of "offloading" TLS compression belongs firmly to the 1990s. As you saw in my benchmark, the percentage difference is small, but the complexity and latency are both lower.

To correctly handle HTTPS offload, web frameworks have to "pick up" the X-Forwarded-For and X-Forwarded-Proto headers. This needs additional config or code in many frameworks, including ASP.NET Core. I.e.: https://learn.microsoft.com/en-us/aspnet/core/host-and-deplo...

If you forget, the result is a redirect loop. By "you" I mean a developer working for a company that isn't the one trying to deploy the code behind NGINX. This happens to me every few months, where Random Product(tm) doesn't work properly because it requires HTTPS despite being behind a HTTPS ingress solution.

No big deal you say, just add the setting and move on? Bzzt... now you've allowed headers to be injected into your applications by random end-users out on the internet, spoofing source IP addresses in your logs, etc...

So now your web app code must be aware of your authorised reverse proxy servers. This also has to be wired up, managed, and set in a config file somewhere.

You now also have a new point of failure, a new location that needs performance tuning, scaling, etc...

Fundamentally, a web server ought to be able to stream static content from memory cache about as fast as the wire can handle it. In which case every "hop" you add also has to have the same throughput! If your web server farm scales to 10 servers of 1 Gbps each, then your reverse proxy must scale to 10 Gbps, or equivalent.

For 'n' layers, the usable fraction of your total available bandwidth drops to 1/n!

Take a typical cloud-hosted Kubernetes solution with a web front end talking to an API tier (god help me I've seen too many of these!), and you could end up with 10+ layers, for 10% efficiency. E.g.:

Cloud load balancer -> Kubernetes Ingress -> Kubernetes Service -> Kubernetes NAT -> NGINX pod -> ... 3x ...

If you've ever wondered why modern apps "feel slow" despite theoretically great throughput... now you know.