Slacker News Slacker News logo featuring a lazy sloth with a folded newspaper hat
  • top
  • new
  • show
  • ask
  • jobs
Library

Comment by leumon

15 hours ago

So 89.4% on Terminal Bench 2 but only 19.1% on Tbench 4. Opus 5 is 89.1%/51.8%.

2 comments

leumon

Reply

aszen  11 hours ago

I was thinking the same, obvious suspicion is they benchmaxed it on older bench.

  • the_duke  42 minutes ago

    Google has always done quite a lot of benchmaxing for Gemini.

Slacker News

Product

  • API Reference
  • Hacker News RSS
  • Source on GitHub

Community

  • Support Ukraine
  • Equal Justice Initiative
  • GiveWell Charities