← Back to context

Comment by BrokenCogs

15 days ago

Every release I see a new benchmark with a new frontier. These graphs are just meaningless now

In particular, where cost is the x axis, a well-funded company with a good but not best model can discount their new model until it reaches the frontier. In some sense this is as much a statement about the efficiency of markets in the presence of public measurement-based comparison and public list pricing as it is about research pace.

you can give it a meaning if you see the relavivity between models. no need to disregard the stat at all.