Comment by nullbyte 20 hours ago 82.7% on Terminal Bench is crazy 3 comments nullbyte Reply toephu2 18 hours ago Is it? There are 5 other models near ~80% and it was achieved in March... which in AI-world seems like a century ago.https://www.tbench.ai/leaderboard/terminal-bench/2.0 ejpir 17 hours ago those are not verified. I've tried forgecode and I cannot believe they didn't do something to influence the benchmarks GodelNumbering 17 hours ago Yup, they were found to be sneaking the answer key using agents.mdhttps://debugml.github.io/cheating-agents/#sneaking-the-answ...
toephu2 18 hours ago Is it? There are 5 other models near ~80% and it was achieved in March... which in AI-world seems like a century ago.https://www.tbench.ai/leaderboard/terminal-bench/2.0 ejpir 17 hours ago those are not verified. I've tried forgecode and I cannot believe they didn't do something to influence the benchmarks GodelNumbering 17 hours ago Yup, they were found to be sneaking the answer key using agents.mdhttps://debugml.github.io/cheating-agents/#sneaking-the-answ...
ejpir 17 hours ago those are not verified. I've tried forgecode and I cannot believe they didn't do something to influence the benchmarks GodelNumbering 17 hours ago Yup, they were found to be sneaking the answer key using agents.mdhttps://debugml.github.io/cheating-agents/#sneaking-the-answ...
GodelNumbering 17 hours ago Yup, they were found to be sneaking the answer key using agents.mdhttps://debugml.github.io/cheating-agents/#sneaking-the-answ...
Is it? There are 5 other models near ~80% and it was achieved in March... which in AI-world seems like a century ago.
https://www.tbench.ai/leaderboard/terminal-bench/2.0
those are not verified. I've tried forgecode and I cannot believe they didn't do something to influence the benchmarks
Yup, they were found to be sneaking the answer key using agents.md
https://debugml.github.io/cheating-agents/#sneaking-the-answ...