Comment by _pdp_

There are downsides depending on how good is your harness. Switching the model is easy enough. Ensuring that the harness continues working the way it did is a completely different thing. This is not just about the prompts but also general behaviour around the model and its infrastructure.

So while it is not complicated and certainly something that can be solved, it is not plug and play.

That being said, we switch to open weight models earlier this month and the results has been more than positive so far. The cost savings are also hard to dismiss.