Comment by a34729t

I've seen lots of successful large k8s installs. This same argument from the nomad crowd comes up constantly. There's a reason why nomad lost, and it's not because it's better.

> Ever hear of a large K8s migration going well

...well, AI infra stacks are more or less k8s/cloud native, especially on the inference side. Nvidia GPU operator plus KubeFlow makes deploying models easy, and I did that manually as well using just the ollama, llama.cpp containers and especially vllm operator. I'm not sure about what's the measuring notation of "large" is, but OpenAI had a blog post about how to manage 7500+ nodes Kubernetes (https://openai.com/index/scaling-kubernetes-to-7500-nodes/). That's 5 years ago and I speculate it will only be even more.

> completely re-implemented under the hood for anything halfway serious

...an example please? I'm building a platform to deploy hundreds of open source apps on Kubernetes, you know, the boring thing you can do with a VPS and maybe use Docker Compose to start -- with an ergonomic twist. I relied on so many features of what Kubernetes and its ecosystem provided, especially with persistent volume, volume snapshot, CRDs and cron job, and all of it is just composed from open source and cloud native software...I'm not sure if that sounds "half way serious" to you.

I know those boring monolithic apps can be done with a VPS, as I exactly came from that background, and I know what the shitty points of having just a VPS are. You don't have a clean control plane, you don't have HA, you don't have distributed storage, and you have to be super aware of the apps that you are deploying with Docker Compose.

What you would think "completely re-implemented under the hood for anything halfway serious", perhaps it means that you don't want to get in the fuzz to manage the complexity and prefer to trade for a far simpler but more primitive solution, and that's totally fine.

Pro tip: Codex seems to love generating great Helm charts so much for some reason unknown. I tried GPT-5.4 high in codex and it easily beats Opus 4.7 max on my own internal helm chart generation benchmark evaluation, which measures HA, deployment and app-specific probes. I've given source code to both Codex and Opus and research about the app config structure, and Codex did so well to just generate secret key-value and convert it to JSON and environ, while Opus insisted on using a dynamic generator at runtime.

> Nomad was cool though

Ahem. I used to deploy Nomad too, but I found it too underwhelming, especially with the networking side of thing.

Consul is perhaps one of the most hated thing in my entire career that I would call it bullshit. I know that later Nomad has its own Raft mode, but the Consul brain rot to me is already very sickening such that I don't want to touch Nomad no more. Vault is fine though, but I prefer OpenBao now.

> Dart is also complete shit: Speed of javascript with verbosity of Java

I don't get where the "speed of javascript" is from. Dart/Flutter is JIT on dev mode, and AOT compiled on production, is that you didn't get the compiler options right?