Comment by functionmouse
6 hours ago
Jokes on them, I don't trust benchmarks
Once something becomes a benchmark it is no longer a good benchmark.
6 hours ago
Jokes on them, I don't trust benchmarks
Once something becomes a benchmark it is no longer a good benchmark.
No, it's when a benchmark becomes a target. You might have a private benchmark that you tell no one about. Would you not trust it?
Good benchmarks are costly to build even for mid-large corporations. And once the benchmark is used on models you really can’t tell if the problems would be scrapped for training
I was speaking in general, not just about AI.