Comment by RJIb8RBYxzAMX9u
7 hours ago
In case this anecdote is not made up, I would implore you or your friend be a bit more subtle at tokenmaxing (ugh). At $JOB, I'm under the same mandate, and it turns out that every prompt is logged and aggregated. When someone else at $JOB asked the team PM who's in charge of the logging, s/he replied that the log is only used to correlate with commits, and nothing else, trust us (wink). I doubt this is unique to my $JOB.
Therefore, sigh burn those tokens, but make sure your prompts are at least superficially defensible, in the unlikely event that you get audited. Use multiple models for the same prompt / task, for instance. It's well know that LLMs are prone hallucinations, so it's only prudent to double / triple cross-check the results with multiple models.
The situation is insane at the company my friend works at. There's no central oversight, because, as I understand it, someone in leadership had the idea that they didn't want to prescribe which AI tools their engineers should use. So they just let their engineers expensify any tools they want. Afaik he hasn't yet hit the upper-bound; last month he said he reimbursed a ~$500 anthropic bill. The same company also makes their engineers go to the Apple store to buy laptops, with their own money, then reimburse it. The credit card points must go crazy over there.
But definitely yeah, normally: be careful about these things. In his case when I said "admin dashboard" i moreso meant the general idea of admin oversight; he's said he's been complimented internally about how much he's using AI.