Comment by mdasen

It also depends on how many tokens it needs to burn through to accomplish something.

At this point, I always look at things like Artificial Analysis' total cost to run their tests. It'll take into consideration the cost of tokens, how many tokens it burns through, and how effectively it uses caching (and the price of that caching).

If a model "costs the same" but its reasoning ends up going through a ton more tokens, it doesn't really cost the same in real world usage.