Comment by ianm218
15 days ago
Why do these benchmarks often show models running on Max i.e. Opus 5.5 (max) is on the frontier? I would think it is never really economical to run max
15 days ago
Why do these benchmarks often show models running on Max i.e. Opus 5.5 (max) is on the frontier? I would think it is never really economical to run max
By definition, the model with the best performance will be on the pareto frontier (regardless of cost).
No. Pareto frontier is cost performance tradeoff.
What jonas21 is stating is correct. If you have the absolute best performance, you are automatically on the Pareto frontier because you’re the most cost effective one (or the only model) on that performance tier.
4 replies →
Why is this downvoted? From my understanding dominated solutions aren't on the paerto frontier e.g. in this case where argon and opus 5.5 are tied the lower cost argon should be the end of the curve.
From the graph it's a flat line, but looking at the point yields a negligible increase with no error bars (indeed if you look at fable 5.1 it's measured as lower than opus and argon). This just seems like bad data or blatant favoritism at worse.
Yikes.
Check out slopalytics. You can draw a pareto line there with max reasoning excluded.