Comment by mrothroc

I have the same experience. I've been running sequential agents in my own harness that is a standard SDLC pipeline (plan, design, code, build, test). It has gates between each stage to control quality.

The big benefit of automating this for so long is that I have lots of data. I analyzed it and found that I can change the models out without much of a change in the output quality.

For one-off tasks, where there is no harness and you're just YOLOing with the TUI, yes, big difference. You need a harness.

The pipeline controls the quality far more than the model, empirically.