Comment by la_fayette

I can recommend Apache Camel (https://camel.apache.org) for similar data integration pipelines and even agentic workflows. There are even visual editors for Camel today, which IMHO make it extremely user friendly to build any kind of pipeline quickly.

Apache Karavan: https://karavan.space/ Kaoto (Red Hat): https://kaoto.io

Both are end 2 end usable within vscode.