Comment by sokoloff
13 hours ago
Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively?
I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero/negative net value, don’t do the first half; if it has net value, don’t do the second half.)
> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively?
Spending $10k and then balking at an additional $2k can make sense to me based on the circumstances. It's an additional 20% that hasn't been budgeted for.
The question is partly about what that 20% gives the employer - for some businesses, you might see diminishing returns from additional productivity of a software developer. E.g. I did some work creating BI reports and warehousing for a manufacturing company. Being more productive just meant faster reports or more reports. But faster/more reports didn't necessarily equate to more money for the business.
Sometimes it's a question of a business being penny wise and pound foolish, but sometimes it does make sense to me to not spend this additional money.
This sounds roughly similar to, "If eating steak is nutritious and tastes good, why would you not eat an 8oz sirloin for breakfast, lunch, and dinner for an entire month?"
"Too much of a good thing" does exist. Someone might feel that it takes time for their organization to "metabolize" the rapid additions made by coding agents. That seems quite reasonable to me. Your comment seems to argue a false dichotomy.
Because there are diminishing returns somewhere for sure. Will the developer be more effective when he gets $2,000 more in tokens? I can't max the $100 subscription. The same for any other employee-related expense. It's a no-brainer to give the developer a solid laptop. But if you'd pay extra $2,000/month in computer and office supplies costs, will it save the employee a few hours? I don't think so. Their salary doesn't matter that much.
I think in this future they don't think a software engineer is making that much, maybe a nice 50k a year, 65K after 10+ years on the job
Still doesn’t make sense if you believe that AI is making a meaningful boost to productivity.
If you are the kind of person who thinks AI will drive down the wages of software engineers that much, you have to think that it is at least a 2x multiplier.
$65k salary all in for a company is easily $100k. So spending anything less than $100k would be what you would aim for.
I honestly don't know how 100k is not effectively the new 40k from late 90s atm in US. 50K is barely enough to do anything.
I mean median household income in 1999 was $40k and median household income now is $84k.
Forward thinking employers should use these limits with a holistic perspective. It's obviously time to take a deserved coffee break if you've token maxxed too much. Maybe take Friday off, AI now allows us to accomplish in four working days what used to take five.
What about a $10k to $20k bill? Or better, 100%-200% of the employees salary? At a certain point in time, finance is going to care greatly about the ability to predict costs. Pre AI, predicting the cost an engineer was quite easy. Salary + benefits + licenses for software. CI systems that were unbounded were rolled up into opex and treated as a separate line item. Where does token spend land in this now?
Finance teams above all else value predictability. Token spend is the opposite of predictable. Eventually these two forces are going to collide and some part of the system will break. My guess is it's token spend, not finance allowing unbound spend on a previously predictable part of the balance sheet.
Set a limit, train people to use tokens efficiently. For anyone getting high value from it, raise their limit.
How do you figure out they are "getting high value from it"
What if they feel like they are getting value from it, but are actually not?
If I had to pay $10k salary for a certain job previously, but I now have to pay $10k + $2k for the same job. Why would I not care about that? You were doing your job perfectly well previously, why do you want more budget?
> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively?
No idea, but companies refusing to buy a 2000$ laptop once every 3 years, a new mouse, keyboard, or screen for < 500$ are very common, more common than the other type.
So I'd expect those to not suddenly change into "let's waste money" companies.
this not making any sense to you is exactly why companies are implementing these caps: you lack the business acumen to understand how financial resources should actually be deployed.
Please consider rewriting this in a way that does not seem to unnecessarily attack parent. You are not wrong about the incentives, but you may not be right about the execution of the restrictions. I should know. I am living through it.
you're fully correct and I apologize for my tone. similarly to many here, I got hit by the caps and frustrated by how many times coworkers were unintentionally or not burning through these resources.
Claiming you have no idea why the managers had to do this felt to me like avoiding any responsibility
1 reply →