Comment by yogthos

Thanks, glad to hear the harness is actually doing its job with smaller models on your end. There definitely seems to be a limit of how small a model can get before it can't do any practical work.

I find I tend to view agentic coding similarly to a genetic algorithm. The model is the mutator function, and the harness along with the tests acts as the selection function. Each round the model generates some plausible code, it gets tested against the constraints, the model gets feedback and iterates on it until it converges on something that's workable. So, the real trick is to make sure the environment is producing correct pressures to guide the model in the needed direction.

Another interesting project in this space I can recommend checking out is ATLAS https://github.com/itigges22/ATLAS