Comment by sroussey

I made a harness too, and recently added eval with opencode and pi and I sat between them. No gaming other than trying to figure out why I was really bad on some evals (I was not returning thinking tokens which broke cache, for example, and different model families do things way different here). I have yet to try my harness for coding but I do have a few terabytes for it to read. And I’d rather use a local model that’s fast and good at retention, but that run could take a year, lol.