Comment by bastawhiz
7 hours ago
Well it's not a benchmark, and it's not really representative of...anything except volume of research and what gets publicized. This mostly just measures how much testing each company does on models with relaxed guardrails and then talks about it. I'm not sure what kind of conclusion you can draw from that. Meta might have the most evil models but if they're piddling around not testing it, they won't ever find themselves with a "high score."
Exactly. It currently seems to be a ranking of how much safety testing each company does. It's also only ever going to be the companies that publicly disclose it happening. (in the case of hugging face, OAI's hamd was forced to disclose)
There's a good way and a two worse ways that companies could optimise this benchmark.