Comment by ranger_danger 5 days ago Which others? 4 comments ranger_danger Reply benaduggan 5 days ago There are several public benchmarks for decision models where they don't top every single one. https://benchmarkheaven.com/jev-models/allI guess their cheeky "Warning: AI Benchmarks, don't trust them" is their way out of that one. woadwarrior01 5 days ago The way that site charges a fee[1] for benchmarking models, makes any benchmarks on it very sus, IMO. TBF, the benchmarking site is probably more profitable than TypeSafe, just like the joke about how WinRAR is more profitable than OpenAI[2]. :)[1]: https://news.ycombinator.com/item?id=49797811 Maxious 5 days ago https://huggingface.co/spaces/multimodalart/jev-decision-ind... is also ranking just for the love of the game.Given that benchmarks are useful, why not give them economic value? 1 reply →
benaduggan 5 days ago There are several public benchmarks for decision models where they don't top every single one. https://benchmarkheaven.com/jev-models/allI guess their cheeky "Warning: AI Benchmarks, don't trust them" is their way out of that one. woadwarrior01 5 days ago The way that site charges a fee[1] for benchmarking models, makes any benchmarks on it very sus, IMO. TBF, the benchmarking site is probably more profitable than TypeSafe, just like the joke about how WinRAR is more profitable than OpenAI[2]. :)[1]: https://news.ycombinator.com/item?id=49797811 Maxious 5 days ago https://huggingface.co/spaces/multimodalart/jev-decision-ind... is also ranking just for the love of the game.Given that benchmarks are useful, why not give them economic value? 1 reply →
woadwarrior01 5 days ago The way that site charges a fee[1] for benchmarking models, makes any benchmarks on it very sus, IMO. TBF, the benchmarking site is probably more profitable than TypeSafe, just like the joke about how WinRAR is more profitable than OpenAI[2]. :)[1]: https://news.ycombinator.com/item?id=49797811 Maxious 5 days ago https://huggingface.co/spaces/multimodalart/jev-decision-ind... is also ranking just for the love of the game.Given that benchmarks are useful, why not give them economic value? 1 reply →
Maxious 5 days ago https://huggingface.co/spaces/multimodalart/jev-decision-ind... is also ranking just for the love of the game.Given that benchmarks are useful, why not give them economic value? 1 reply →
There are several public benchmarks for decision models where they don't top every single one. https://benchmarkheaven.com/jev-models/all
I guess their cheeky "Warning: AI Benchmarks, don't trust them" is their way out of that one.
The way that site charges a fee[1] for benchmarking models, makes any benchmarks on it very sus, IMO. TBF, the benchmarking site is probably more profitable than TypeSafe, just like the joke about how WinRAR is more profitable than OpenAI[2]. :)
[1]: https://news.ycombinator.com/item?id=49797811
https://huggingface.co/spaces/multimodalart/jev-decision-ind... is also ranking just for the love of the game.
Given that benchmarks are useful, why not give them economic value?
1 reply →