← Back to context Comment by WhitneyLand 6 hours ago It’s not outdated at all to use tokens to estimate performance, it’s directly related. 2 comments WhitneyLand Reply prism56 2 hours ago But why should I care? If my metrics are speed and cost? How many tokens it takes as a user is arbitrary to some extent. WhitneyLand 37 minutes ago If speed is a metric for you, tokens required to solve a problem affects that metric.All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.
prism56 2 hours ago But why should I care? If my metrics are speed and cost? How many tokens it takes as a user is arbitrary to some extent. WhitneyLand 37 minutes ago If speed is a metric for you, tokens required to solve a problem affects that metric.All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.
WhitneyLand 37 minutes ago If speed is a metric for you, tokens required to solve a problem affects that metric.All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.
But why should I care? If my metrics are speed and cost? How many tokens it takes as a user is arbitrary to some extent.
If speed is a metric for you, tokens required to solve a problem affects that metric.
All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.
Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.