Comment by tedsanders
2 hours ago
Sounds like arena.ai: https://arena.ai/leaderboard
People vote on pairs of model responses and they aggregate everyone's votes into a leaderboard.
One thing to be aware of that the distribution of prompts on arena.ai can be pretty different than real-world usage. The people who spend time on that website casting votes tend to be more interested in entertainment than professional productivity. And turn-based voting can punish useful behavior like asking clarifying questions instead of guessing at assumptions.
Yes, I've seen Arena.ai and the reasons you cited are why I suggested something better that's more private, personalized and shareable internally without being skewed and gamed.