Comment by tonyhschu

fwiw I work on data ingestion pipelines and I've found that starting with just boxes-and-arrows in something like Excalidraw gets you 80% of the way to knowing what you actually want. The gap between "I can picture it" and "I can build it on a webpage" is mostly a d3 learning curve problem, not a design problem.

xyflow that the creator mentioned is probably the right call for pipeline DAGs though -- we use it internally for visualizing our scraping workflows and it was surprisingly painless to get running

Sort of like this? https://docs.tecton.ai/docs/introduction/interactive-tour I used https://github.com/xyflow/xyflow for this, with css animations for the edges. It’s probably easier now with coding agents and what not