Comment by jaggs
1 month ago
It's a great benchmark. Don't listen to the haters. This one is especially interesting.
https://aibenchy.com/compare/anthropic-claude-sonnet-4-6-med...
1 month ago
It's a great benchmark. Don't listen to the haters. This one is especially interesting.
https://aibenchy.com/compare/anthropic-claude-sonnet-4-6-med...
This one's even more interesting
https://aibenchy.com/compare/anthropic-claude-opus-4-6-mediu...
Who knew Anthropic was this far behind???
Yeah, but actually that's not a good look. Anyone who's used Gemini will know how random it is in terms of getting anything serious done, compared to the rock solid opus experience.
Their benchmark is chock-full of things like that: It's deeply flawed and is essentially rating how LLMs perform if you exert yourself trying to hold them entirely the wrong way.