Comment by ernsheong
9 hours ago
I built my harness in pi within herdr, I cloned (zipped and downloaded) 0xRichardH/pi-herdr-subagents and went from there, and used pi to build itself, adding gate workflow state control, provider fallbacks (I use many token plans), subagent skill injection, etc.
It is highly custom to my needs and wants, and I think every developer needs to do this. I only talk to my planner which plans, and it subs out to designer, oracle, coders, testers, and reviewers, etc. It is thus highly optimized for correctness. You can TDD or no TDD. You can fast track small changes. I tweak my harness dozens of times as I encounter new edge cases (esp when I switch models and encounter models not as good at following instructions).
As you can start to see, it is better to own the harness because nobody can build something custom that 100% fits your needs or development philosophy.
I cannot agree more. We all have our own needs, and it’s never been easier to make something that meets them exactly.
I have been running my own setup and evolving it since November as I come up with more ideas and needs. The nicest bit is not being tied to one provider or another. Anthropic decides to stop supporting Claude agent sdk with subs? Or astra outstrips other models? My harness now works with Claude/cursor/codex and soon ACP. It didn’t start that way, it was originally all in on Claude, but now all the things I built on top of Claude now work with the others too.
I did a similar thing, took those sub agents primitives and ran with it: https://github.com/durdn/herdr-interactive-subagents I do still use more than one harness though.
Correct me if I'm wrong, but the harness will always be dependent on the underlying model, and useless without it. All custom harnesses are being built, could be obsolete in the next big-generation-jump of the models.
I might be absolutely wrong, but "harnesses" / cc-derivatives became "good enough" only maybe a year ago max. Before that, people were pushing for gigantic folder structures with custom documents and "pretend you're X" stuff.
My point is, depended on what you're trying to achieve, testing out current-gen harnesses, and nudging your workflows towards them might be better RoI, rather than chasing something that might be throwaway code a quarter later.
Obviously, this really depends on whether you believe model development will speed up or slow down in the upcoming future.
I switched between several local and remote providers and models and over different API (anthropic/openai) and it worked fine, just some minor issues but they were fixed within an hour.
And the system prompt worked great regardless, so i don't think your main point holds, especially as models improves; it isn't throwaway code, but for sure it's evolving constantly, as my own workflow keeps changing.
> My point is, depended on what you're trying to achieve, testing out current-gen harnesses, and nudging your workflows towards them might be better RoI, rather than chasing something that might be throwaway code a quarter later.
Fair point, depends if it's an hobby or you are a developer full time, in the latter case i think it's definitively worth it.
You are wrong. I honestly am having trouble understanding how you think these things work, and what you believe a harness is in the first place.
There is the standard API that things like openRouter exposes, thats supported by every provider, and all it is, is manipulating a large blob of memory by adding your own words or asking the GPU to append new words.
Everything else around it is window dressing. All providers accept that API, (only Anthropic has blocked access on their consumer subscription tier).
Its how businesses write their 'bots'. Nobody - except the people trying to sell you "magic" - is incentivized to remove that kind of access.
I can switch out openai/anthropic/deepseek/openrouter/kimi/selfhosted at the drop of a hat. Its a big reason you'll see a comment on HN talk about "There is no moat".
Calling them cc-derivatives is wrong. I've seen many features land in other harnesses long before they came to Claude.
The only thing Claude seems to be "cheating" with is that:
- They provide Windows users a unix VM hosted on their servers - I dont want or need that
- They're better 'trained' at doing compaction / providing the illusion of continuity
- They might do automatic model-switching (not sure if they do) - not something i need.
Sure, if you're in the market for magic then dont put time in having your own harness and just accept the lock-in. Since I'm using them so much i think its worth the investment.
Sorry, I didn’t mean to imply that what you’re doing is wrong, although re-reading my own comment sounds like so.
The capabilities of a model unlocks certain harness behaviour, and in its terms might be beneficial to automate more of your x, y, z.
I guess computer use would be an example I was thinking about. Certain models are not as good as some current models (Astra/Fable) in understanding through screenshots, or going through some hoops in some environments. As models evolve, they unlock new capabilities, that you either have to keep implementing in your harness, or using an existing one. I’m pro-using existing just because it doesn’t make sense for my org to support another dev tool internally, when there is heavy better development happening by people who focus on that.
The reason why i said cc-derivative, as far as I can see, cc was the one that convinced people “it might be a good idea to dev this way”. Sure you had some lesser known harnesses around, but a year+ ago, nobody really cared.
I don’t think, at least as of now, anything is really much of a lock in. We switch out between cc/codex/cursor and it takes trivial amount of time to set stuff up, depended on how your dev loop goes.
Again, it really depends on one’s conviction of “how fast things will develop from now on”. I personally think stuff will keep changing very fast. In a year what we think of harnesses will be different. Investing in custom tooling might be bad, as that is basically a lock-in. But again, I don’t know the future, but that’s my bet. Hovering around the edges and using what others battle test is an easy way out right now.
Also in some larger orgs, there is more friction in model choices. So you slowly start losing the benefits of model-swapping. Obviously different for personal projects.
1 reply →
A lot of what a custom harness does isn't necessarily interacting with a model, it's the framework around it. Part of harness engineering is figuring out what can be done deterministically to avoid calling an LLM. A generational jump doesn't obsolete those areas, it just means you get the right answer (maybe) faster and easier when you do call the model.
There's a difference between their harness and your harness. The latter can be tuned to your preferences, while the former cannot. A custom harness can do everything that a packaged harness cannot. Good luck getting Claude Code to coordinate with multiple model providers or deeply integrate with your environment.
The point you're making is to be a consumer, which admittedly, is probably best for most people, but it's not going to satisfy the hacker. Normies aren't building custom harnesses, so your argument is DOA for the target demographic.
Also, vendor lock-in. No thanks.
We might have different views on where the SE as a profession is headed, I think. Again, I might be wrong, nobody knows the future, but we didn’t even use the word “harness” that much a year ago. There’s a decent chance in two years the harness itself as a concept might be obsolete. Depended on how fast your teams move, investing in a custom harness might be a bad resource allocation. Unless, obviously you’re in top tier companies which can benefit from it and pivot fast.
Again, this is all subjective risk assessment. I very much support any engineer to learn how to make harnesses, and I, myself, have a long way to go. We’ll see where we are in a year or two, I guess.
I also built a custom harness tailored to my needs using pi.dev underneath. From time to time I use the 'official' harnesses of openai/anthropic but can't confirm that they are much better adapted to that harness.
100% every professional developer at some point should build its own harness as daily driver
I'm very interested in this but I am confused on what Pi provides you if you are building the harness? What does Pi get you that writing from scratch doesn't?
Any good starting points or tutorials you recommend?
Pi is just a nice base and it has defined extension protocols and such. You might as well start there, it's just easier and going from nothing to working to adding whatever functionality is like 2 minutes.