Comment by seer
15 days ago
Hmm how does your custom harness compare to Claude’s?
I’ve tried the various alternatives (opencode, pi.dev) but for “have several physical machines, each running an orchestrator and 5-10 agents, and each agent spawning its own subagents to do work in parallel, all of that on a tanky GitHub project boards” level of work only Claude was doing work dependably.
I can have a few orchestrator leads which I talk to and brainstorm, and let them file tickets, organize them and assemble teams of agents to work of the various problems. And all of that with just a smidge of custom tooling on top of Claude Code’s default.
Plus don’t have a budget to spin all of this on cloud runners so I made sure to do as much as I can with local work on several machines I own. Was quite easy to achieve with Claude.
Do you have something similar built on top of a open source harness, or is it from scratch?
Well I work at BigCo and I have no idea how it compares to Claude Code because I’m not allowed to use Anthropic models outside of our custom company harness. Opus 5.5 works extremely well, it can launch multiple subagents, assign them tasks, use our company’s inscrutable build system (that no one really understands anymore), and run tests, all by exposing harness commands in the system prompt. I don’t think we are lacking in any way.
The switch in mindset I experienced was to go from "Working on a feature" to "curating a board" to "overseeing angets curating boards".
So that I'm not just relying on one or a few agents to accomplish a task. I work on the whole product and just drive the direction of tickets and their "what done looks like" requirements, and let the machine organize things on its own, usually 10-20 independent agents (each spawning their own subagents). I think i might have capacity to handle more, just don't have the subscription tokens to fuel it and the budget for the hardware to run it.
Haven't worked in BigCo so no idea how that compares to their tooling. I'm sure it works much more efficiently and on a bigger scale, just being curious about how it all plays out.
I built http://github.com/brokkai/mjolnir/ to do exactly this cross-harness over ACP. As a bonus it can let Opus drive Luna as a subagent, which is just ridiculously cost effective. (5x clst multiplier means Haiku is much more expensive for realistic task sizes.)
how do you see which agent on which machine is waiting on you? or do you check each orchestrator by hand
Please tell me how you do that without Claude forgetting its context all the time. I feel like luke fucking skywalker talking to r2d2.
claude: Help me, Obi-Wan Kenobi. You're my only hope.
me: who is obi-wan kenobi
claude: oh you have to log in again.
me: <logs in>
me: play the whole message
claude: what message?
To be honest I’m not sure - the tooling around it has processes to start things, and gourds around tool usage to remind Claude of proper procedure.
But ultimately I find it a silver lining more than a problem. I am building and changing tooling so fast, and the models tooling itself is changing so fast, that by the time I establish a workflow, I either A) have solved the original problem so no longer needed, B) have built other tooling that distorts the picture, or C) the labs have released something new and cool that changes the equation drastically so your previous setup is gone anyway.
For example I built some procedure on how to split work between fable and opus and sonnet to optimise token usage, then opus 5.5 came along and made all of this obsolete.
Not per “token usage” mind you, opus is still expensive, but more “effort per task time finish” kind of metric. If I want to close things faster while still being relatively within budget, it is just easier to send all to opus - it will now just solve thigns quicker so you still use similar amount of tokens / money, but quicker, which is a trade I’ll make any day.
And every time “the system is working” I either just stumble onto the next resource constraint, or I feel uncomfortable and start refactoring to make things better and inevitably destabilise the whole “machine”