← Back to context

Comment by palata

8 hours ago

I really like the SourceHut CI, because:

1. When the build fails, you can SSH into the machine and debug it from there.

2. You can super easily edit & run the manifest without having to push to a branch at all. That makes it super easy to even try a minimum reproducible example on the remote machine.

Other than that, self-hosting (with Github or preferrably Forgejo) makes it easy to debug on the machine, but then you have to self-host.

Self-hosted runners with Github is a whole world of pain because it literally just runs commands on the host and does not handle provisioning/cleanup, meaning you need to make sure your `docker run` commands don't leave any leftover state that can mess up future/concurrent builds. It doesn't even handle concurrency by itself, so you have to run multiple instances of the runner.

  • Github supports ephemeral runners which are limited to a single job.

    You can use `workflow_job` webhook to be notified of a new job, after that you need to call `generate-jitconfig` API to get a just-in-time configuration token, and then you can start a Github runner in ephemeral mode with the JIT token.

    This allows you to orchestrate Docker containers, KVM instances, etc., which are used for a single time, and then destroyed.

    There are some open source projects, such as using ephemeral Kubernetes pods with the ephemeral runners.

  • That was included in my "but then you have to self-host" :-).

    As I said, I really like the SourceHut CI.

    • Do you have any experience self-hosting SourceHut? I’d really like to do so, but I get weak knees every time I look at the docs for it.

      1 reply →