Dogpile was only a good idea while Search Engines were mostly trash.
You needed to search all of them to find something decent.
That's roughly analogous to today. Ignoring cost, you'd be way better off asking all the LLMs to solve a problem (like coding) where you can verify the answer.
So the question is, for things like that -> can a group of models perform better than frontier models, especially at a reasonable cost?
Fable is not a great value, so unless you're trying to find answers to Erdos questions, you can probably do better on cost.
You can probably typically ask 3 or 4 of the top Chinese models for an answer and get a response for the same Fable question... Given that Fable isn't that much better, it's not surprising you can do better for a large subset of problems.
Our whole stack is radically open source — frontend and backend alike, Apache-2.0 licensed — and so is everything behind this benchmark. That is how a benchmark number earns trust: verifiability, not hype.
Mixture of Models, perhaps? tbf to OP, the setup they're proposing has also been recently evangelized by other "ai gateway" products (like OpenRouter, JusCode, Fireworks etc), so there's likely something useful here.
Good ideas are usually still good across time and tools
Dogpile was only a good idea while Search Engines were mostly trash.
You needed to search all of them to find something decent.
That's roughly analogous to today. Ignoring cost, you'd be way better off asking all the LLMs to solve a problem (like coding) where you can verify the answer.
So the question is, for things like that -> can a group of models perform better than frontier models, especially at a reasonable cost?
Fable is not a great value, so unless you're trying to find answers to Erdos questions, you can probably do better on cost.
You can probably typically ask 3 or 4 of the top Chinese models for an answer and get a response for the same Fable question... Given that Fable isn't that much better, it's not surprising you can do better for a large subset of problems.
> Dogpile was only a good idea while Search Engines were mostly trash.
Precisely.
3 replies →
Dogpile was only a good idea while Search Engines were mostly trash.
Well, search engines are trash again. Perhaps it should come back
1 reply →
There is no reason a smarter model will not build an internal smarter/efficient router itself
Exactly
High quality, fast & cheap (all 3 combined) - is a formula success.
It’s just way easier said than done.
There's a reason most pros will tell you pick two of the three.
ensemble models always did the best at Kaggle
we did the same: https://trustedrouter.com/blog/prometheus-2-new-draco-state-...
You need update your blog posts.
The repos have since been moved to BUSL-1.1: https://github.com/Lore-Hex/quill-router/commit/8155ac666ae0...
everything has been updated
Mixture of Models, perhaps? tbf to OP, the setup they're proposing has also been recently evangelized by other "ai gateway" products (like OpenRouter, JusCode, Fireworks etc), so there's likely something useful here.
Ensemble Methods:
https://en.wikipedia.org/wiki/Ensemble_learning
This sounds like networking.
Did OP invent an “intelligence router?”
This is the don't use the same EC2 size for everything on your service approach.
Well, you can try to make triangular wheels, but they are round for a reason
Indeed. What a deep cut
metacrawler.com
timecube?