← Back to context

Comment by dgxyz

7 hours ago

Yeah this. As I explain many times to people, processes are the only virtualisation you need if you aren’t running a fucked up pile of shit.

The problem we have is fucked up piles of shit not that we don’t have kubernetes and don’t have containers.

Containers are just processes plus some namespacing, nothing really stops you from running very huge tasks on Kubernetes nodes. I think the argument for containers and Kubernetes is pretty good owing to their operational advantages (OCI images for distributing software, distributed cron jobs in Kubernetes, observability tools like Falco, and so forth).

So I totally understand why people preemptively choose Kubernetes before they are scaling to the point where having a distributed scheduler is strictly necessary. Hadoop, on the other hand, you're definitely paying a large upfront cost for scalability you very much might not need.

  • Time to market and operational costs are much higher on kubernetes and containers from many years of actual experience. This is both in production and in development. It’s usually a bad engineering decision. If you’re doing a lift and shift, it’s definitely bad. If you’re starting greenfield it makes sense to pick technology stacks that don’t incur this crap.

    It only makes sense if you’re managing large amounts of large siloed bits of kit. I’ve not seen this other than at unnamed big tech companies.

    99.9% of people are just burning money for a fashion show where everyone is wearing clown suits because someone said clown suits are good.

    • Thanks. You’ve reassured me that I’m not going mad when I look at our project repo and seriously consider binning the Dockerfile and deploying direct to Ubuntu.

      The project is a Ruby on Rails app that talks to PostreSQL and a handful of third party services. It just seems unnecessary to include the complexity of containers.

    • I have a lot of years of actual experience. Maybe not as much as you, but a good 12 years in the industry (including 3 at Google, and Google doesn't use Docker, it probably wouldn't be effective enough) and a lot more as a hobbyist.

      I just don't agree. I don't find Docker too complicated to get started with at all. A lot of my projects have very simple Dockerfiles. For example, here is a Dockerfile I have for a project that has a Node.JS frontend and a Go backend:

          FROM node:alpine AS npmbuild
          WORKDIR /app
          COPY package.json package-lock.json .
          RUN npm ci
          COPY . .
          RUN npm run build
      
          FROM golang:1.25-alpine AS gobuilder
          WORKDIR /app
          COPY go.mod go.sum .
          RUN go mod download
          COPY . .
          COPY --from=npmbuild /app/dist /app/dist
          RUN go build -o /server ./cmd/server
          
          FROM scratch
          COPY --from=gobuilder /server /server
          ENTRYPOINT ["/server"]
      

      It is a glorified shell script that produces an OCI image with just a single binary. There's a bit of boilerplate but it's nothing out of the ordinary in my opinion. It gives you something you can push to an OCI registry and deploy basically anywhere that can run Docker or Podman, whether it's a Kubernetes cluster in GCP, a bare metal machine with systemd and podman, a NAS running Synology DSM or TrueNAS or similar, or even a Raspberry Pi if you build for aarch64. All of the configuration can be passed via environment variables or if you want, additional command line arguments, since starting a container very much is just like starting a process (because it is.)

      But of course, for development you want to be able to iterate rapidly, and don't want to be dealing with a bunch of Docker build BS for that. I agree with this. However, the utility of Docker doesn't really stop at building for production either. Thanks to the utility of OCI images, it's also pretty good for setting up dev environment boilerplate. Here's a docker-compose file for the same project:

          services:
            ui:
              image: node:alpine
              ports: ["5173:5173"]
              working_dir: /app
              volumes: [".:/app:ro", "node_modules:/app/node_modules"]
              command: ["/bin/sh", "-c", "npm ci && npx vite --host 0.0.0.0 --port 5173"]
            server:
              image: cosmtrek/air:v1.60.0
              ports: ["8080:8080"]
              working_dir: /app
              volumes: [".:/app:ro"]
              depends_on: ["postgres"]
            postgres:
              image: postgres:16-alpine
              ports: ["5432:5432"]
              volumes: ["postgres_data:/var/lib/postgresql/data"]
          volumes:
            node_modules:
            postgres_data:
      

      And if your application is built from the ground up to handle these environments well, which doesn't take a whole lot (basically, just needs to be able to handle configuration from the environment, and to make things a little neater it can have defaults that work well for development), this provides a one-command, auto-reloading development environment whose only dependency is having Docker or Podman installed. `docker compose up` gives you a full local development environment.

      I'm omitting a bit of more advanced topics but these are lightly modified real Docker manifests mainly just reformatted to fewer lines for HN.

      I adopted Kubernetes pretty early on. I felt like it was a much better abstraction to use for scheduling compute resources than cloud VMs, and it was how I introduced infrastructure-as-code to one of the first places I ever worked.

      I'm less than thrilled about how complex Kubernetes can be, once you start digging into stuff like Helm and ArgoCD and even more, but in general it's an incredible asset that can take a lot of grunt work out of deployment while providing quite a bit of utility on top.

Hahhah, yuuuup.

I can maybe make a case for running in containers if you need some specific security properties but .. mostly I think the proliferation of 'fucked up piles of shit' is the problem.

Disagree.

Different processes can need different environments.

I advocate for something lightweight like FreeBSD jails.