Comment by fock

For reasons I recently got acquainted with enterprise scheduling solutions. IBM/HCL workload automation specifically - the company is actively looking for a replacement but nothing except for the chinese sponsored open source comes close it seems to me (and that is a step too far for management...).

The thing you miss with scripting is that a good workflow engine essentially provides a simple and well-defined layer allowing you to couple many concurrent processes and providing ressource management. Curiously out of the box all the opensource workflow tools do so too - like you said people (Airflow has pools, but thrn ...) essentially built very weird DSLs to do high-level composition and there you can indeed rather easily code it yourself. I assume the companies originating these have integrations for the other bits too while everyone else does fake elastic scaling with some halfbaked workarounds. And from time to time someone comes up with a new DSL and some GUI - ignoring all the hard parts.