Comment by llm_trw

> we only have months to years of experience with the mistakes that LLM's and other AI's make.

The mistakes are also very much model dependent. That you have build a system which improves the accuracy of one models output give you no confidence that it will work on even the next generation of the same model.