Comment by zambelli

Nice! I'm not surprised at your findings (anymore). Mechanical reliability is the key to small models, and it's a big unlock. I've seen the same thing you just described. And the agnostic nudges forge sends at inspired by exactly that. Just show the model how it failed, gracefully, and it'll likely figure a way out of it itself.

Forge doesn't have a SWE-specific eval, but I've built a custom coding harness (not public yet but maybe soon) built on forge and saw the same behavior you seem to have seen in agentic coding.