Comment by ebiester
2 days ago
Missing here: some organizations were rewarding high token usage as productivity without critical evaluation. People were afraid to be in the bottom because outcomes weren't being measured.
It is a giant Goodhart's law lesson
Give your agent a perfectly working code, insist that the output is not what it should be. Go to lunch. By the time you come back, the poor thing will evaporate a small lake trying to figure it out.
"i'm in aisle 32 of the data centre. please evaluate the previous query using exclusively servers 2438-2458. and quickly, it's f-ing freezing in here".
What!? Companies rewarding high token usage? That's inane, insane, and small brained. Who in their right mind equivocates spending more money to bring more productive. I'll just set up some burn jobs to kill tokens unnecessarily and then everyone else will too and the company will go bankrupt in 10 days. It seems inconceivable for a company to set up a "who can spend the most of our money" leaderboard for any other context
I have friends at two different companies that are taking a stick, rather than carrot, approach to this. They've set monthly minimums for token usage. Anything less than that gets you dinged in your next performance review. Imagine hiring a carpenter and writing a bad online review for them because they didn't use their hammer enough, even though the end product was on time, on budget, and worked well.
I was at a company 20 years ago that took this approach to automated tests. Everyone must write 2 a day, even if that's the only code they write that day. Once it was clear that this was being checked with automation, scripts were going around to generate and commit tests that 1 + 2 == 3 (replace with random numbers). Of course tokens are being burned this way at companies like this.
I think a better analogy would be "didn't use enough nails", since it's consumables. To which the response would be "nailgun. pop. pop. pop-pop. pop-pop-pop. pop. 'Those damn squirrels sure can move'".
Go look up "Tokenmaxxing."
Yes, it's as stupid as it sounds.
This is essentially companies making their engineers use LLMs as much as possible, and if you don’t, you go on a pip. Many such cases.
If you think this qualifies as insane, you really haven't met many managers, have you...
there are boards… endless boards… ranking by token usage :)
Given that all of AI is built around the premise that whoever sets fire to the most money wins, it's just users following the lead the vendors.