Comment by tokioyoyo
4 hours ago
Sorry, I didn’t mean to imply that what you’re doing is wrong, although re-reading my own comment sounds like so.
The capabilities of a model unlocks certain harness behaviour, and in its terms might be beneficial to automate more of your x, y, z.
I guess computer use would be an example I was thinking about. Certain models are not as good as some current models (Astra/Fable) in understanding through screenshots, or going through some hoops in some environments. As models evolve, they unlock new capabilities, that you either have to keep implementing in your harness, or using an existing one. I’m pro-using existing just because it doesn’t make sense for my org to support another dev tool internally, when there is heavy better development happening by people who focus on that.
The reason why i said cc-derivative, as far as I can see, cc was the one that convinced people “it might be a good idea to dev this way”. Sure you had some lesser known harnesses around, but a year+ ago, nobody really cared.
I don’t think, at least as of now, anything is really much of a lock in. We switch out between cc/codex/cursor and it takes trivial amount of time to set stuff up, depended on how your dev loop goes.
Again, it really depends on one’s conviction of “how fast things will develop from now on”. I personally think stuff will keep changing very fast. In a year what we think of harnesses will be different. Investing in custom tooling might be bad, as that is basically a lock-in. But again, I don’t know the future, but that’s my bet. Hovering around the edges and using what others battle test is an easy way out right now.
Also in some larger orgs, there is more friction in model choices. So you slowly start losing the benefits of model-swapping. Obviously different for personal projects.
> The capabilities of a model unlocks certain harness behaviour, and in its terms might be beneficial to automate more of your x, y, z.
Not really.
Sure its nice for a model to nativly be trained on understanding browsers & DOM/js model, so it can effectively use a `browser` tool.
A model needs a certain 3d mental model before it can successfully use a `blender` tool. Opus 4.5 had this to some extend without specifically training for it - but sure this is one aspect i think models will rapidly improve on this year.
But at the same time these tools carry a description of how they're used specifically - so changing out the `blender` tool for some other 3d modeling software shouldn't affect its capabilities. Using/creating a coherent description of the desired art-style is most of the value of the work.
So its still not clear to me what you mean with "certain harness behavior" or "unlock new capabilities".
You get access to a model that can be better or worse at certain tasks, sure.
The big change i've noticed last year is they've gotten better at prompting themselves (subagents, delegate, w/e you want to call it).
But the explosion of how fast things went last year is mostly "existing software * better model". For that capability explosion to continue you'd need ever more "software" that improve with AI (or more existing software unblocking their interface for model use)
There will definitely be more software, but you need time to incorporate it in your work (like eg `git`) for it to meaningfully feels like AI is unlocking new things when using it.
For new stuff like being able to control a robot dog, we first all need to have a robot dog for it to feel like very fast development.
----
As for "Computer use" specifically that seems like a bad example from my perspective.
The only real "added capabilities" over the past year i've seen is better selfprompt + comprehending screenshots.
Every other use case i'd consider computer-use is just friction that needn't have been there in the first place. Its the software/os you're using that doesnt have a proper interface that's creating the friction. Not the lack of model/harness capabilities.
I have had a `browser` tool for a year that's just a debug-socket to ff/chrome with eval + pre/post screenshot, and ever since 4.4 ~ 4.5 every model has understood it just fine and can do basically anything.
Reasoning _what_ to do with it is the 'intelligence' we're trying to benchmark and everything since Opus 4.5 was basically 'good enough' for anything non-creative a human could do.
E.g. having a model get good at creating drafts of emails based on support tickets, is a function of how good the cli/mcp and instructions are.
Can you be more specific in what capability increase you see getting unlocked?
As a final point, the models are so good now that you can ask claude inside claude to write/setup plugins for pi to do everything claude can do.
The reason i dont use pi to ask to improve the claude harness, is because anthropic is trying to build a moat with their closed source crap. They'll succeed because enterprise is slow to adapt and doesn't understand its wasting money, not because their models unlock certain harness behavior. The competition who is not wasting money will have a step up.