Comment by 0xDEAFBEAD

From a big-O notation perspective, if you're serving N queries, working the kinks out of your routing system costs perhaps O(logN) in developer time, whereas LLM provider savings grow as O(N).

In other words, the more queries you're serving, the more worthwhile it looks to figure out a viable method of model routing.