← Back to context Comment by nrhrjrjrjtntbt 4 days ago Why measure in minutes and not tokens? Seems you could cheat by slowing the ai down. 1 comment nrhrjrjrjtntbt Reply wmf 4 days ago They measure the time it takes a human to complete the task. They don't care how long the AI takes (although in practice it's much faster than human). Measuring tokens isn't a good idea because newer models can complete tasks using fewer tokens.
wmf 4 days ago They measure the time it takes a human to complete the task. They don't care how long the AI takes (although in practice it's much faster than human). Measuring tokens isn't a good idea because newer models can complete tasks using fewer tokens.
They measure the time it takes a human to complete the task. They don't care how long the AI takes (although in practice it's much faster than human). Measuring tokens isn't a good idea because newer models can complete tasks using fewer tokens.