Comment by david-gpu
11 hours ago
That may be the case, but simply running the same code review twice with the same model may give very different results as well. Impossible to know with the information we have.
11 hours ago
That may be the case, but simply running the same code review twice with the same model may give very different results as well. Impossible to know with the information we have.
yeah, there is definitely still a lot of vibes to all this still, I'm working on evals next to learn and leverage, to reduce the vibes component to the claims
today we can use some anecdata based on our observations, this also helps shape the dimensions we want to eval the agents-tasks on