Comment by supermatt
3 months ago
Probably naive questions:
Does that also mean that Gemini-3 (the top ranked model) loses to mistral 3 40% of the time?
Does that make Gemini 1.5x better, or mistral 2/3rd as good as Gemini, or can we not quantify the difference like that?
Yes, of course.
Wow. If all the trillions only produces that small of a diff... that's shocking. That's the sort of knowledge that could pop the bubble.
I wouldn't trust LMArena results much. They measure user preference and users are highly skewed by style, tone etc.
You can litteraly "improve" your model on LMArena by just adding a bunch of emojis.