Comment by BoorishBears
2 hours ago
... why didn't you link to the actual benchmark which does have Jev at the top?
https://benchmarkheaven.com/jev-models
You linked to some weird subtable that labeled: " Not the default — not the JevBench Score", that can only be reached after you see what I just linked... lmao are you really this hard up about things?
Also every single question (even in the hard set) is single dimensional?: https://github.com/fstandhartinger/jevbench/blob/main/datase...
Jeeze, this is getting sad. I guess after all the mass-psychoses where people thought pointless things are going to change the world, we were due for a mass-psychosis where something interesting just has to be pointless?
Because I was specifically responding to your claim that Jev’s training recipe would give it better accuracy than others. It doesn’t have better accuracy than others. You could do as well or better by distilling qwen for example.
Jev is ranked higher than others on the overall benchmark due to speed and/or cost, not accuracy.