Comment by iTokio
6 months ago
Isn’t that limited to a single node?
How would you configure a cluster? I’m trying to explore lightweight alternatives to kubernetes, such as docker swarm, but I think that the options are limited if you must support clusters with equivalent of pods and services at least.
I've found you can get pretty far with a couple of fixed nodes and scaling vertically before bringing in k8s these days.
Right now I'm running,
- podman, with quadlet to orchestrate both single containers and `pods` using their k8s-compatible yaml definition
- systemd for other services - you can control and harden services via systemd pretty well (see https://pyinfra.com/) to manage and provision the VMs and services
- Fedora CoreOS as an immutable base OS with regular automatic updates
All seems to be working really well.
> Isn’t that limited to a single node?
Yes. Though unless you have a very dynamic environment maybe statically assigning containers to hosts isn't an insurmountable burden?
> How would you configure a cluster?
So, unless you have a service that requires a fixed number of running instances that is not the same count as the number of servers, I would argue that maybe you don't need Kubernetes.
For example, I built up a Django web application and a set of Celery workers, and just have the same pod running on 8 servers, and I just use an Ansible playbook that creates the podman pod and runs the containers in the pod.
In the off chance your search didn't expand to k3s, I can semi-recommend it.
My setup is a bit clunky (having a Hetzner cloud instance as controller and a local server as a node throught Tailscale), from which I get an occasional strange error that k3s pods fail to resolve another pod's domain without me having to re-create the DNS resolver system pod, and that I so far failed at getting Velero backups to work with k3s's local storage providers, but otherwise it is pretty decent.
K3s is light in terms of resources, but heavy in operational complexity, I’m not looking for a smaller version of kubernetes but for a simple way to run container backed services when you’re not google but a small company, something that has few moving parts but is very reliable and low maintenance.
I've been back and forth on this for a long time, but I've just decided at this point that I either settle for podman or docker on a single host, or go to Talos / k3s / k8s. There's a lot of tools there, a lot of inertia, and eventually it's likely that I will need to solve the problems that k8s does.
YMMV
It that's what you want to do, just get something running... what's wrong with docker swarm?
1 reply →
HashiCorp Nomad is probably the only real alternative. It's what in using, and I like it better than the overcomplexity of k8s.
I've been reading and watching videos about how you can use Ansible with Podman as a simpler alternative to Kubernetes. Basically Ansible just SSHs into each server and uses podman to start up the various pods / containers etc. that you specify. I have not tried this yet though so take this idea with a grain of salt.
whew, "alternative" is doing a lot of work there.
Contrast:
With
If you don't happen to have a cluster autoscaler available, feel free to replace the for loop with |head -1 or a break, but I mean to point out that the overall health and availability of the system is managed by kubernetes, but ansible is not that
>> lightweight alternatives to kubernetes
microk8s seems exceedingly simple to setup and use. k3s is easy as well.
I once tried Nomad for a very brief moment. Not sure if it fits your bill.
Nomad is weird. Its OSS version is like a very limited trial of paid version. At least last time I tried it. To a point that it was more productive for me to install k3s instead.