Comment by merb

6 months ago

most of the time the "far more complex setup" is mostly easier than the reimplementation of kubernetes with ansible.

the irony here of course is that Kubernetes is sort of designed to be run on bare-metal.

As a life-long ops person, a lot of the capabilities of kubernetes were things we used to bolt-on. (Like logging a boot ID so that we could track looks easier, service managing, having as little local state as possible so that the workloads are somewhat immutable and nodes can be cycled out, draining machines due to hardware failure (or cordoning them at least) etc;etc;etc)

Cloud came in and made some of those things a little easier I guess, but the value of kubernetes for most people is that they don't have to care about it because the providers give you a one size fits all solution: just don't peek under the hood.

Even the best implementation of Kubernetes (GKE) has a huge amount of waste due to software that has been added to every node and configured wrong.

Sure, but for the most part these places just didn't use either. You can go extremely far without any of the things the kubernetes gets you. (And if you want to, you can get some if those things for a lot lower cost than k8s)

I mean, you can just install kubernetes on your own system. Most of these things that AWS provides exist as services. You get some benefit from AWS running them, but you pay in literal dollars and the invisible complexity they add.