← Back to context

Comment by reissbaker

19 hours ago

We do both: managed Kubernetes when it's available (AWS, Nebius, others), but for some hardware vendors they just give us raw machines and we self-host K3s on their nodes. We're an open-source LLM inference company so we're basically always scrambling for GPUs wherever we can get them, which means we need to be fairly scrappy with what we support while still having a semi-sane interface for ourselves internally. Kubernetes makes that pretty easy: onboarding a new vendor takes ~minutes, and then everything Just Works and we can interact with the pool of compute the same way we do every other pool since the K8s API is standard, with all of our built-in prod monitoring tools immediately set up and running.

That being said I love exe.dev and have been a happy customer since launch. It's a different use case but they do an amazing job at it. Very, very easy personal cloud dev box. But K8s is very very good too, just for production workloads rather than personal ones!

This makes a lot of sense and is like the perfect use case for programming towards a "standard api".

I run it at home and at work, and while I do hate installing it, once that part is done I've never run into these problems that people claim requires a 20 person(!) team to baby sit it. Maybe my scale is too small or whatever, but its hard not to think that maybe they are just "holding it wrong"...