Comment by _pdp_
6 hours ago
We have our own... so I was making the comment form our own experience working with the model.
It is a lot trickier to use kimi compared to sonnet - hence why it seems that sonnet is more powerful while I think it is down to the harness.
How did you make your own harness, I am curious to know more about the building process and please feel free to share your harness.
If someone were to not use your harness and rather use some stock harness though, what is the one that you would recommend? I am curious about that too.
I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc.
As I am writing this I am fixing a bug in the harness that could cause infinite loops under some conditions.
For simple interactions looping over the the llm complete function is not really that difficult. Put some tools, write the loop and exit.
It starts getting more tricky when you need to detect cycling behaviour, when some tools might need to abort but do so gracefully, when you need to wait for something to complete while allowing other parts to continue, when you want to protect against too much usage after you hit certain thresholds, when you need to retry whatever, when you need to compact or truncate to maintain good context window and do so with the model in mind (kimi), when messages need to be structured in certain ways to handle various model capabilities, etc.
Pi.dev, Hermes and all the other I have seen do not do even half of the things I have enumerated above - not at least out of the box.
> I really don't mean to discourage anyone but I find that making your own harness is pretty complex process. We where very naive when we started, thinking this is just a loop, but then it turned out it has all kinds of nuances, edge-cases, things to be considered, etc.
I'd be interested in your harness if its open source, please share some more resources
> Pi.dev, Hermes and all the other I have seen do not do even half of the things I have enumerated above - not at least out of the box.
Interesting that they don't do these things out of the box, do you still have some ideal setup then though? Like Pi.dev + X/Y/Z thing which can make things work close enough to ideal.
Because although our conversation is interesting, I feel like I am unable to take any enforcable action and I might default back to opencode. So I would love if you could talk more about it.
3 replies →