Comment by Ameo

3 years ago

Networking is the place where I notice how tall modern stacks are getting the most.

Debugging networking issues inside of Kubernetes feels like searching for a needle in a haystack. There are so, so many layers of proxies, sidecars, ingresses, hostnames, internal DNS resolvers, TLS re/encryption points, and protocols that tracking down issues can feel almost impossible.

Even figuring out issues with local WiFi can be incredibly difficult. There are so many failure modes and many of them are opaque or very difficult to diagnose. The author here resorted to WireShark to figure out that 50% of their packets were re-transmissions.

I wonder how many of these things are just inherent complexity that comes with different computers talking to each other and how many are just side effects of the way that networking/the internet developed over time.

6 comments

Ameo

Reply
parasubvert  3 years ago

Kubernetes has no inherent or required proxies or sidecars or ingresses, or TLS re-encryption points.

Those are added by “application architects”, or “security architects” and existed long before Kubernetes, for the same debatable reasons: they read about it in a book or article and thought it was a neat idea to solve a problem. Unfortunately, they may not understand the tradeoffs deeply, and may have created more problems than were solved.

imglorp  3 years ago

There's been a highly annoying kubectl port-forward heisenbug open for several years which smells an awful lot like one of these dark Go network layer corners. You get a good connection establish and some data flows, but at some random point it decides to drop. It's not annoying enough for any wizards to fix. I immediately thought of this bug when Nagle in Go came up here.

https://github.com/kubernetes/kubernetes/issues/74551

anthk  3 years ago

Wireshark exists since forever.

  • Beltalowda  3 years ago

    Wireshark doesn't tell you anything about what's wrong with your code. It just tells you "yup, the code is doing something wrong!"

    Figuring that out in Kubernetes ... yeah, good luck with that.