The problem of the sota models clamping down on third party harnesses should be stated upfront. Getting a SOTA model in a custom harness requires API pricing or risking an account ban, correct?
This preliminary change in cost may not make it worth it for people depending on which is their "favorite agent". Especially as the default harnesses for said agents continue to improve...
> The problem of the sota models clamping down on third party harnesses should be stated upfront. Getting a SOTA model in a custom harness requires API pricing or risking an account ban, correct?
The opposite, how easy it is nowadays should be clarified. Codex's app-server quite literally is built to be integrated with, authentication is a breeze and it's quite stable, compared to the TUIs that are around.
It's a real shame Codex is moving in the direction of hiding stuff on local disks from users, hopefully they're revert the decision of encrypting the agent>sub-agent prompts so we get back introspection again...
Presumably people want to hear the opinions of other people and still believe that they might learn more from the their experience than by exclusively interacting with AI models?
The alternative of that first impulse should be what exactly? Telling AI to research Medium AI drivel articles (since nobody seems to be posting long/medium form content anywhere else these days) to figure out what’s the best approach to building an AI harness?
BTW just my personal observation but I generally find AI notoriously bad at building any AI applications compared to standard software (which is not surprising given the scarcity of high quality training data)
Not the parent commenter, but most of it is surprisingly simple. You basically start with a "chat app" where you have a list of messages, send the whole conversation to an LLM and it replies back, which also gets added to the same list.
And you add a small twist, that instead of a 1-to-1 back and forth, you instead put it into a loop, where the LLM reply can itself "have a turn", e.g. a tool invocation, where your system is the one that replies (e.g. with the tool invocation's result). That's pretty much it, you have a 1 to potentially many "chat".
The harder part is getting all the "soft" parts right, like how to have well-behaving tool calls, timeouts, prevent huge cycles eating up tokens, but there are no one way to solve these, it's a fundamentally heuristic-heavy area.
It's pretty simple nowadays if you know conceptually how they work. Running the LLM calls in a loop with tools is an agent. You only need 10 or so basic tools to accomplish nearly anything, and you can build a dynamic skill system from that. Look at https://github.com/patw/pengy, ignore the app look at the spec.md file, feed that to your current agent of choice and make your own version. Use whatever tech stack or UI you're comfortable with. Change some of the choices in how it works, so it fits what you want to work.
This comment is a great example of how large and strange the skills gap in AI is right now.
Curious why your first impulse is not simply to point your favorite agent at a few examples and start brainstorming/planning from there?
Multiple times I’ve built a purpose specific bespoke tool starting this way. In fact, it’s a great way to learn how specialized tools are built.
The problem of the sota models clamping down on third party harnesses should be stated upfront. Getting a SOTA model in a custom harness requires API pricing or risking an account ban, correct?
This preliminary change in cost may not make it worth it for people depending on which is their "favorite agent". Especially as the default harnesses for said agents continue to improve...
> Getting a SOTA model in a custom harness requires API pricing or risking an account ban, correct?
No, only Anthropic has that policy (and I think even that is relaxed for an unknown period if you use the Claude Agent SDK: https://support.claude.com/en/articles/15036540-use-the-clau...).
OpenAI, Kimi, Qwen, GLM and Deepseek all allow it.
I'm not sure about Gemini.
> The problem of the sota models clamping down on third party harnesses should be stated upfront. Getting a SOTA model in a custom harness requires API pricing or risking an account ban, correct?
The opposite, how easy it is nowadays should be clarified. Codex's app-server quite literally is built to be integrated with, authentication is a breeze and it's quite stable, compared to the TUIs that are around.
It's a real shame Codex is moving in the direction of hiding stuff on local disks from users, hopefully they're revert the decision of encrypting the agent>sub-agent prompts so we get back introspection again...
Presumably people want to hear the opinions of other people and still believe that they might learn more from the their experience than by exclusively interacting with AI models?
The alternative of that first impulse should be what exactly? Telling AI to research Medium AI drivel articles (since nobody seems to be posting long/medium form content anywhere else these days) to figure out what’s the best approach to building an AI harness?
BTW just my personal observation but I generally find AI notoriously bad at building any AI applications compared to standard software (which is not surprising given the scarcity of high quality training data)
This was my first thought as well. I can read online review about a product but I rather ask my friends their experience with it. Same thing here.
Not the parent commenter, but most of it is surprisingly simple. You basically start with a "chat app" where you have a list of messages, send the whole conversation to an LLM and it replies back, which also gets added to the same list.
And you add a small twist, that instead of a 1-to-1 back and forth, you instead put it into a loop, where the LLM reply can itself "have a turn", e.g. a tool invocation, where your system is the one that replies (e.g. with the tool invocation's result). That's pretty much it, you have a 1 to potentially many "chat".
The harder part is getting all the "soft" parts right, like how to have well-behaving tool calls, timeouts, prevent huge cycles eating up tokens, but there are no one way to solve these, it's a fundamentally heuristic-heavy area.
It's pretty simple nowadays if you know conceptually how they work. Running the LLM calls in a loop with tools is an agent. You only need 10 or so basic tools to accomplish nearly anything, and you can build a dynamic skill system from that. Look at https://github.com/patw/pengy, ignore the app look at the spec.md file, feed that to your current agent of choice and make your own version. Use whatever tech stack or UI you're comfortable with. Change some of the choices in how it works, so it fits what you want to work.
Implementing your own agent is very easy. Here is a minimal agent in 60 lines of Python without dependencies:
https://github.com/99991/MinimalAgent
You only need a single tool to start with. All recent LLMs know how to use bash for reading/writing/editing/executing.
Why would the language matter? Just use your favorite.
Look at pi coding agent.