← Back to context

Comment by xlayn

20 hours ago

If you use Qwen3.8 27B to manage, to orchestrate, do do shallow research, and then make it ask claude you will notice that Qwen can get out of Haiku, Sonnet and Opus almost as good answers... I was researching how good the model asnswers where, I can't get to work with anything but Fable, but Qwen can get good work from any of them. My theory is that if you make Qwen understand you, Qwen can elaborate as much as needed to get any of those models get good results, because it's filling in details. So using Qwen to direct claude also get's you amazing results way way cheaper...

Update: let me elaborate more, I read the HN entry about the team of 5 Qwens being close to fable quality... so I embarked to add to my harness (https://github.com/alainnothere/privibe because we all have one right?) and once I had added the orchestrate tool I had to now measure somehow it it's able to improve quality, so I claude and then Qwen to run a experiment to call my harness passing the model to use and ask them to do something, 3 rounds I think and see what quality came out of it... and as per the grading... qwen and claude agreed that the quality of the response comming out the harness using the orchestrate was VERY CLOSE, with opus and fable providing more "in my opinion you should" which matches my experience on why they are better.

One more update: I can't get copilot models to work decently, it may be my way of working... but I can get Qwen to follow what I try to do... if Qwen get's me he can then fire copilot cli and get amazing results by using copilot... bonus points seems like copilot cli is cheaper with the AIC (cost of the requests) so you can make the copilot subscription longer

46 comments

xlayn

Reply
drkleiner  19 hours ago

I do the opposite of this, use Qwen3.8-27B as subagents, Opus 5.5 as orchestrator and planner, never implementer, and it cuts my token usage significantly. So it is using Claude to direct Qwen. Never thought of the other way around, will give this a try soon as possible.

  • bluegatty  15 hours ago

    Both of those orchestrations will work for sure.

    Smart agent to orchestrate cheaper agents for labour.

    Dumber agents to be very explicit about asking smarter agents for requests that require deep information.

    This is a thing - the SOTA models really light up the more information you can provide them, and humans are too lazy / incapable of doing that in every prompt.

    Entirely depends on what you are trying to do.

    Edit: you can also use a smart agent to ask another smart agent, obviously, which is good for personal workflows.

    • fireant  6 hours ago

      Doesn't that muddle intent of the original prompt? If the smarter model didn't understand your intent correctly, the dumber one sure won't and it rephrasing and expanding the prompt will just amplify it's misunderstanding, no?

      2 replies →

  • beefsack  12 hours ago

    This is how I do it too. I run three levels:

      * Orchestrator: Opus 5.5
  * Feature lead: GPT 6.1 Sol
  * Worker: Muse Spark 1.3 Contributor

    As long as you have an appropriate project and sandbox for Muse Spark it's the real secret sauce for my setup and it's dirt cheap on OpenCode Go.

    GPT 6.1 Sol is surprisingly efficient too, I've been impressed with it.

    My orchestration skill setup is https://github.com/beefsack/processed-beef/

    • nextaccountic  9 hours ago

      Just one thing

      Does anyone have any evidence that having the most powerful model being the orchestrator, and other sub agents being progressively less powerful, is optimal? Is this just common sense or it has been measured somehow?

      For me it could be that this scheme may make the code itself poorly written, verbose or faulty

      2 replies →

    • bluerooibos  5 hours ago

      And which agent checks the work that the Worker does? If it's garbage then isn't that just extra tokens consumed by the Orchestrator or Feature lead?

      I'd be interested to know what the difference in cost is in this setup on building a complex feature vs using Opus directly.

  • jeef_berky  17 hours ago

    My goal was/is to have a local model as a "stable" part of the system, or at least a part of it that you have full control over, and have it reach out to the cloud for some Opus. In practice, to get results, I operate as you say.

    At this point if I could do anything with zero effort or time, I would get Opus 5.5 to make a grand scheme, handoff to the local model, and give it ability to reach out to Opus again.

    • sixdimensional  13 hours ago

      I do something like this by using old rule-based bot tech as the first layer you hit, and if it hits any situation it does not understand deterministically, it shells out for help to an external "smarter" agent.

      That way you can do old school intent matching and rule-based bots that work super well, and only shell out for deeper work.

  • ihuman  19 hours ago

    What do you use to get Opus to call Qwen?

    • unrented7977  16 hours ago

      I just fell Claude "there's an LLM server on this port serving this model, use it as a subagent"

      Claude will write some scripts to call the LLM API and use it like any other tool.

      I haven't found a pressing need for anything more complex than that. My multi-agent setups tend to be a pile of bespoke scripts per project anyway, an MCP server is only gonna get in the way. But I suppose this is because I self-host the models, if you're using OpenRouter or whatever it may need more complexity than a curl call to an unauthenticated port

      1 reply →

    • hocusPokeUs  17 hours ago

      So I use Google Gemini to plan or review; otherwise same loop so brain dumping how I get Gemini to Qwen/Gemma/Meta's model I forget the codename for; local agent reads the AI Studio log locally via Google Drive where they store the logs by default in a JSON format

      So with the gdrive app for my OS it just looks like a local file path. Had the local agent learn to parse it.

      I still prompt in AI studio direct as needed; local agents see the log change but cannot push back to Gemini their result.

      Few reasons:

      AI Studio is available where ever via browser, no VS Code or my usual vim setup to miss. I can switch between projects there as an idea hits and the agent switches up the local repo path and session history.

      AI studio can search web, etc, etc already so my agent is less complicated with addons in general

      Also using AI Studio between local agent turns has been my "human in the middle" approach; to have an explicit gap in the process and check each turn.

      I only have a 24GB 3090 most models also take longer locally and I never hit the free tier limits with Gemini :D

      Though the latest Meta model is twice as fast at 30B for me as other similar size models. 1-2 hour code gen and QA of build errors is now 30-50 minutes to parse latest changes from Gemini and get a successful build. The Meta model is pretty great at "fill in the middle" from a well structured plan.

      Overall this loop feels like a normal workday pace. I freelance so I have a lot more say over that.

    • drkleiner  19 hours ago

      Custom MCP with a variety of functions, I have different Qwen profiles with a variety of subagent numbers and cache pools, my default is 192k pool with 64k initial context that can grow 32k at a time up to 128k, allowing 3x64k, 2x96k or 1x128k and 1x64k subagents. Qwen uses Pi harness. Model I use specifically is ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF

      4 replies →

    • cjonas  15 hours ago

      you can easily run qwen locally in a pi harness (or similar) and just tell claude code to delegate all implementation work to it. claude will even setup the harness with the local model

    • verdverm  19 hours ago

      You can have a setup like this with OpenCode agents, where you can define custom top-level agents, subagents, and assign specific models to each.

      https://opencode.ai/docs/agents/

      There are a lot of ways to do this if you stop using Big Ai harnesses that make it difficult to use models besides theirs.

    • nullstyle  19 hours ago

      ohmypi will spawn subagents across different providers out of the box. Opus will spawn subagents running on my local ds4 installation just fine

koolba  20 hours ago

> My theory is that if you make Qwen understand you, Qwen can elaborate as much as needed to get any of those models get good results, because it's filling in details.

How is this different than just telling the model you’re using to fill in the gaps itself?

If you’re not manually reviewing the intermediate hop, how does that add to the “understanding you”?

  • xlayn  19 hours ago

    there is no difference... each case may be different, I have a local qwen gpu setup... and is fast... but not as fast as claude. Fable is amazing and if it cost the same as my qwen setup I'll be running fable and never had qwen in first place. But Fable is crazy expensive, Qwen is not as fast but cheaper... Haiku is fucking dumb IN MY HANDS... BUT my local qwen is quick enough to be briefed, get the idea and delegate. I work with Fable mostly delegating to opus/sonnet and it's cheaper than just fable, I can't stand working with opus or the other models.

    Why Qwen directing the others? For the things claude refuse, Qwen can make very specific questions and still get an answer without triggering the thing.

    • brabel  4 hours ago

      I am using Haiku 5 and it’s amazing for the cost. I almost always use it instead of Opus5.5 for simpler tasks. In my experience it’s much better than the Qwen models!

  • make3  18 hours ago

    I think that his idea is that a lot of preparation / setup work is boiler plate / brain dead stuff that can be done by a shallower model.

    It's like, find the relevant scientific papers by using Google search and the scientific paper MCPs, summarizing stuff, etc. Then the big model can come in and actually think on the artifacts

    • skeptic_ai  13 hours ago

      Big model can easily delegate to luna to do this task.

      I ask task x -> big model thinks and delegate to luna heavy dumb work.

rao-v  18 hours ago

Oh man I’m been playing with little -> big vs big -> little sub-tasking a bunch.

My sense so far is it’s very much about problem domain “taste” and there is no one correct answer (that is token cost efficient).

Does the smartest agent need the full flavor of the context of the problem to make real progress (i.e what’s been tried, how you’ve influenced the approach) or does the smartest agent really need a crisp problem definition and context room to really work.

  • josht  12 hours ago

    > Oh man I’m been playing with little -> big vs big -> little sub-tasking a bunch.

    I too wonder the same...

    The real-world provides us many examples of such big -> little, or smart -> not-as smart workflows. For instance, we don't typically ask juniors or mid-level SDEs to design solutions to exceedingly challenging problems for a reason. However there are _some_ areas that those with less experience can tackle that is relatively risk-free if their work is reviewed by a more experienced engineer, which leads me to a conclude that it's task dependent.

  • make3  18 hours ago

    I suspect that a lot of the fudging around with MCPs and tools can be delegated to smaller agents. Astra doesn't need to look at 300 websearch results directly or to extract text out of raw HTML

    • rao-v  14 hours ago

      Absolutely no matter whether you generally go big small or small big, you should always be like “ask haiku / Luna to check all X” or “see if there are other ways to Y”

ninininino  19 hours ago

Anthropic has a native solution to this which is that you can use `/advisor` to adjust config to enable using Sonnet but set Opus as an advisor or use Haiku and then set Sonnet or Opus as advisor that the weaker model will automatically call when it thinks something warrants it:

https://code.claude.com/docs/en/advisor

It allegedly will automatically judge when to call the stronger model, not sure how much instructions in prompts or AGENTS.md can steer it.

jimbokun  18 hours ago

Add this to the things I never imagined when thinking about what a world with artificial intelligence might look like.