← Back to context Comment by happyPersonR 17 hours ago more thinking == more tokens === more money LOLL 2 comments happyPersonR Reply overfeed 15 hours ago Os there a cost benchmark out there? I wonder how frontier models are doing over time for cost per problem solved. drob518 15 hours ago I think they are optimizing for one-shot performance because that will drive usage. They can’t afford to look bad in the benchmarks. And if that means consuming an order of magnitude more tokens, well, that’s good for business, too.
overfeed 15 hours ago Os there a cost benchmark out there? I wonder how frontier models are doing over time for cost per problem solved.
drob518 15 hours ago I think they are optimizing for one-shot performance because that will drive usage. They can’t afford to look bad in the benchmarks. And if that means consuming an order of magnitude more tokens, well, that’s good for business, too.
Os there a cost benchmark out there? I wonder how frontier models are doing over time for cost per problem solved.
I think they are optimizing for one-shot performance because that will drive usage. They can’t afford to look bad in the benchmarks. And if that means consuming an order of magnitude more tokens, well, that’s good for business, too.