Comment by jonas21
15 days ago
By definition, the model with the best performance will be on the pareto frontier (regardless of cost).
15 days ago
By definition, the model with the best performance will be on the pareto frontier (regardless of cost).
No. Pareto frontier is cost performance tradeoff.
What jonas21 is stating is correct. If you have the absolute best performance, you are automatically on the Pareto frontier because you’re the most cost effective one (or the only model) on that performance tier.
It is true that the model with the overall best performance is on the Pareto frontier, but it is not necessarily true that a given model's best cost-performance tradeoff is at max reasoning. (IE you could have a model that is on the frontier at medium reasoning, but at max reasoning becomes more expensive and not sufficiently good enough to remain on the frontier.)
2 replies →
Huh, I guess you're right, but I interpreted jonas21's comment in relation to the above comment, which was about all models running on Max (which implies this would be a comment relating to all points on the pareto curve).
Why is this downvoted? From my understanding dominated solutions aren't on the paerto frontier e.g. in this case where argon and opus 5.5 are tied the lower cost argon should be the end of the curve.
From the graph it's a flat line, but looking at the point yields a negligible increase with no error bars (indeed if you look at fable 5.1 it's measured as lower than opus and argon). This just seems like bad data or blatant favoritism at worse.
Yikes.