Comment by pvillano
1 day ago
Wikipedia defines cheating as actions designed to subvert or disobey rules in order to obtain unfair advantages without being noticed. This is the definition I use.
What you've described is not applicable to this definition of cheating.
I care about agents reading the solution from the git history, breaking test harnesses, or using disallowed external tools, because these prevent SWE bench from being an accurate predictor of real world usefulness in fixing bugs
No comments yet
Contribute on Hacker News ↗