Comment by janalsncm
1 hour ago
Someone else posted the jevbench site which compares jev to a bunch of other models. If you look only at the accuracy dimension:
https://benchmarkheaven.com/jev-models?w=100-0-0-0#jevc-weig...
Jev actually isn’t anywhere near the top. It even loses to open weight clones. This tells me that whatever their “calibration” dataset is, it doesn’t seem to be anything special.
... why didn't you link to the actual benchmark which does have Jev at the top?
https://benchmarkheaven.com/jev-models
You linked to some weird subtable that labeled: " Not the default — not the JevBench Score", that can only be reached after you see what I just linked... lmao are you really this hard up about things?
Also every single question (even in the hard set) is single dimensional?: https://github.com/fstandhartinger/jevbench/blob/main/datase...
Jeeze, this is getting sad. I guess after all the mass-psychoses where people thought pointless things are going to change the world, we were due for a mass-psychosis where something interesting just has to be pointless?