Comment by michaelt
5 months ago
Plenty of people have scammed their way to the top of the benchmark league tables, by training on the benchmarking datasets. And a lot of the people who do this just get ignored - they don't take much heat for it.
If the scam hadn't gained enough publicity for people to start paying attention, he would have gotten away with it :)
But not really, which is what confuses the heck out of me. Thousands of people downloaded and used the model. It obviously wasn’t spectacular.
It’s like claiming to have turned water into wine, then giving away thousands free samples all over the world (of water) so that everyone instantly knows you’re full of crap.
The only explanation I can imagine for perpetrating this fraud is a fundamental misunderstanding that the model would be published for all to try?
I just can’t wrap my head around the incentives here. I guess mental illness or vindictive action are possibilities?
Hard to imagine how this plays out.