Comment by jedisct1

I use swival.dev where a decision model (Cloudflare's Clef running locally is free and works great) is used for automatic model selection, but also to complete a task, just like the LLM. It gives confidence scores on the reliability of the LLM output, is used instead of the LLM, even mid-task when appropriate, etc.

And yes, that saves money since less tokens are sent to a remote LLM.