Comment by sams99
5 hours ago
… on one benchmark, that has 2 modes of running. 1 shot and 1 shot with repair. Zero tools.
Then ignore the rules of the bench (which is a pretty obsolete one anyway), give it an agentic loop of sorts via delegation and python eyes and then win.
Sorry but this is not how benchmarking works
No comments yet
Contribute on Hacker News ↗