Comment by 827a
9 hours ago
This too shall pass. Among my software engineering friend group bubble: Every single individual (~12 of us) are actively and seriously tokenmaxing. We have middle-managers who have been given an AI mandate, upper-managers saying "uhh...maybe that brush stroke was too broad" when they look at the bill every month, and zero people in that chain have the authority or even ability to roll it back. This week one of my friends cobbled together an agent that runs in an infinite loop, grabs whatever song they're actively listening to on Spotify, writes it in a file, then instructs the agent to emit tokens for 2-3 minutes on what that song and previous songs that day might mean for that person's mental state, like a little music-based diary. Repeat, run all day, 24/7. Kinda cool. But its just a way to use tokens, because the first thing all these AI labs built was a good coding model, and the second thing they built was a dashboard for admins to track how much their users are using the good coding model.
A TON of companies are getting looted by the AI labs and AI users. Many will not survive. I think Meta will be one of them (a shell of their former selves by 2030). The ones who survive to thrive in the 2030s will be the ones that are relentlessly focused on their customers and products, not the process. If you don't regularly hear both "AI would be awesome for that" and "actually AI probably won't be good for that", your company won't make it. You'll either get lapped by the companies who find the strong use-cases, or you'll get looted by infinite and aimless tokenmaxing. The path through the middle is far more narrow than most companies realize, and some major, major companies are waking up to that harsh reality; for some, too late.
In case this anecdote is not made up, I would implore you or your friend be a bit more subtle at tokenmaxing (ugh). At $JOB, I'm under the same mandate, and it turns out that every prompt is logged and aggregated. When someone else at $JOB asked the team PM who's in charge of the logging, s/he replied that the log is only used to correlate with commits, and nothing else, trust us (wink). I doubt this is unique to my $JOB.
Therefore, sigh burn those tokens, but make sure your prompts are at least superficially defensible, in the unlikely event that you get audited. Use multiple models for the same prompt / task, for instance. It's well know that LLMs are prone hallucinations, so it's only prudent to double / triple cross-check the results with multiple models.
The situation is insane at the company my friend works at. There's no central oversight, because, as I understand it, someone in leadership had the idea that they didn't want to prescribe which AI tools their engineers should use. So they just let their engineers expensify any tools they want. Afaik he hasn't yet hit the upper-bound; last month he said he reimbursed a ~$500 anthropic bill. The same company also makes their engineers go to the Apple store to buy laptops, with their own money, then reimburse it. The credit card points must go crazy over there.
But definitely yeah, normally: be careful about these things. In his case when I said "admin dashboard" i moreso meant the general idea of admin oversight; he's said he's been complimented internally about how much he's using AI.
Is this an ad?
> Many will not survive.
Thankfully the people responsible have already prepared a golden parachute to land safely to destroy something else.