Comment by d2p
1 hour ago
I clicked through and it showed Qwen at the top at 55.4 compared to 55.3 for Opus Max. I have a screenshot.
Then I clicked away and back, and now it goes Qwen second, with 58.4, to Opus Max at top with 59.2.
I have screenshots of both. The description above the chart is the same in boh cases:
> Artificial Analysis Agentic Index > Represents the weighted average of agentic capabilities benchmarks in the Artificial Analysis Intelligence Index (GDPval-AA v2, ³-Banking)
What happened? How can the scores change so much in a few seconds?
They JUST updated their methodology:
https://artificialanalysis.ai/methodology/intelligence-bench...
I have been suspicious of these AI leaderboard sites for some time now, and this only increases that suspicion.
Fixed the result, eh? In both senses of the word.
What was the change?
They should probably freeze the results before publishing.
Welp. That didn't last long
Same, they just updated it. Hacker news effect?