Comment by csomar

I am working on this problem and while I agree with the author, there is room for improvement for the current status quo:

> So going beyond the section title: CI systems aren't too complex: they shouldn't need to exist. Your CI functionality should be an extension of the build system.

True. In the sense that if you are running a test/build, you probably want to start local first (dockerize) and then run that container remotely. However, the need for CI stems from the fact that you need certain variables (ie: you might want to run this, when commit that or pull request this or that, etc.) In a sense, a CI system goes beyond the state of your code to the state of your repo and stuff connected to your repo (ie: slack)

> There is a GitHub Actions API that allows you to interact with the service. But the critical feature it doesn't let me do is define ad-hoc units of work: the actual remote execute as a service. Rather, the only way to define units of work is via workflow YAML files checked into your repository. That's so constraining!

I agree. Which is why most people will try to use the container or build system to do these complex tasks.

> Taskcluster's model and capabilities are vastly beyond anything in GitHub Actions or GitLab Pipelines today. There's a lot of great ideas worth copying.

You still need to run these tasks as containers. So, say if you want to compare two variables, that's a lot of compute for a relatively simple task. Which is why the status quo has settled with GitHub Actions.

> it should offer something like YAML configuration files like CI systems do today. That's fine: many (most?) users will stick to using the simplified YAML interface.

It should offer a basic programming/interpreted language like JavaScript.

This is an area where WebAssembly can be useful. At its core, WASM is a unit of execution. It is small, universal, cheap and has a very fast startup time compared to a full OS container. You can also run arbitrarily complex code in WASM while ensuring isolation.

My idea here is that CI becomes a collection of executable tasks that the CI architect can orchestrate while the build/test systems remain a simple build/test command that run on a traditional container.

> Take Mozilla's Taskcluster and its best-in-class specialized remote execute as a service platform.

That would be a mistake, in my opinion. There is a reason Taskcluster has failed to get any traction. Most people are not interested in engineering their CI but in getting tasks executed on certain conditions. Most companies don't have people/teams dedicated for this and it is something developers do alongside their build/test process.

> Will this dream become a reality any time soon? Probably not. But I can dream. And maybe I'll have convinced a reader to pursue it.

I am :) I do agree with your previous statement that it is a hard market to crack.