Comment by paimapi
4 days ago
to be fair, you don't need to start the y-axis at zero [0] but for some of the graphs where the lowest value is close to 0 the best practice is to do so
there's a fun Excel artifact where it auto-selects the 'relevant' range with no adjustment for how proportionally close to 0 the values are - a professional researcher publishing to a journal should know better (and should be ridiculed for not incorporating best practices) but for a personal blog by an SWE this really isn't the worst sin
[0] https://digitalblog.ons.gov.uk/2016/06/27/does-the-axis-have...
IMO in this case is mandatory to start from 0 because it alters the visual perception.
Just look at the first chart: the distance between Fable 5.1 and Sol is <5%, but it looks like 25 or 30%.
I disagree. The y-axis is some arbitrary intelligence score that we use as a proxy for performance on whatever our specific task happens to be. So it doesn't matter if a model is a 0, 1, or 20 along this axis, they are all useless for the tasks I want.
And as the complexity of your score increases, the cutoff goes up. We can quibble about where your personal cutoff is, but it aint 0.
Y-axis is between 0 and 100.
But even if it was between 0 and Inf+, it still gives you a wrong perspective, especially if you are not paying attention, on model capabilities.