Comment by prog_1
1 year ago
ie when you cant beat them, make new metrics
and you can absolutely evaluate how smart someone is in a 2min casual conversation. You wont be able to tell how well they are in some niche topic, but %insert something about different flavors of intelligence and how they do not equate do subject matter expertise%
It’s a common pattern that AI benchmarks get too easy, so they make new ones that are harder.