Comment by RALaBarge
6 days ago
I work for a router company too, I ran some tests on all of the cheapest models and came to the same outcome where a handful of small models ran together in conjunction outperform SoTA models -- outperforms in that it got a 95% vs a 94% and I bet that changes with the day of the week. Anyways, I did get a similar result in a different sort of measurement.
Can you elaborate on running them "in conjunction"... are you running the same query on multiple models and then using a third model to judge or make consensus? or am I misunderstanding completely. I'd like to understand how these small models "run together"
I do this in OMP, a fork of Pi. It lets you set different models for different tasks. So with an API that has many different companies models I can set the Plan model to the best one, right now I am using GLM 5.2 for that, it plans really well. I have Vision set to Kimi 2.7 Code (cheaper and vision is just fine). Minimax M3 is set to the Advisor role (double checks work). Deepseek v4 Flash is set for the Task role. And MiMo 2.5 pro is set as default.
With this setup GLM handles planning and managing my AGENTS.md, and orchestrating subagents for tasks from the plan/todo GLM created. The tasks themselves are handed off to Deepseek v4 flash to implement with strong instructions and examples for each agent. Minimax M3 reviews the output as the Advisor and recommends changes, catches bugs, and whatnot, subagents can be re-run with that information.
Overall I am saving a lot using some of these smaller models. But with this setup I am getting great results.
Are you using stock OMP or do you have any additional prompts you can share. This looks really promising to me, want to study/learn/copy/steal.... :)
Anything you can share would be useful to me.
2 replies →