Comment by solenoid0937
25 days ago
Also at a FAANG here. Surprised you don't manage to use $300 in a whole day. It's almost trivial to productively use that much in under an hour.
Leadership is not being dumb, at least on this topic. If your token usage is that low, you just aren't using AI that much (even if you think you are.)
I use $30 a day to produce a decent amount of code. Certainly more than we need - thinking about/designing the correct solution/distilling requirements is still the bottleneck. How can you possibly even review $300/day worth of output?
It doesn’t have to be $300/day worth of output tokens. It could be like $290/day worth of input tokens to teach both you and the model about the problem you are solving and then $10/day worth of output tokens.
And what about you knowing the problem and the solution, but are just worrying about the impact downstream. Most of my time is spent managing those. I know the exact code to be published. And some time I already have it committed in my local branch. Then you need to make everyone aware of what it entails and that's usually how you can spend days on a simple bug or a change request.
Software is a big graph of interlocked rules. And if you can grasp the whole or the part you own (and you should be able to), it's often very easy to see the control points. You don't have a coding bottleneck anymore, you have a communication bottleneck[0]. Which is an organizational issue, not anything relevant to engineering.
[0]: See Naur's Programming as Theory Building and Brooke's Mythical Man Month.
It could be thinking tokens or tokens passed in via RAG.
If you give it $290 of input tokens for $10 of output tokens, you are doing something wrong. I.e. you paste the whole CI output into the prompt instead of giving it a link to the file, and then the AI greps its way through it (using a fraction of the tokens).
Sometimes AI overdoes things and it re-runs the whole testsuite because the tail command didn't have enough lines, but the other way round messes up the context so much so that in the end all that context is useless.
I used Claude about a week ago to do a pretty intensive refactoring. Cleanup, initial modularisation, beginnings of a test suite, and better isolated build. In a span of couple of hours, and over a sequence of 20+ new commits, I burned a hair over $100 in tokens.
If you are working on a seriously large legacy code base, I can see how you'd get to >$250 on a bad day.
If you build your own reviewer layer/tool it will burn a ton of tokens. Millions of tokens of input.
You, review bots and first pass bots can chew through tokens. Also if you haven't put effort into your harnesses, the agent will have to spend more time and tokens figuring things out again and again
Use expensive models at high effort
Also you regarding Claude usage limits:
> Before the doomers come in, you get $200 in API credits every month for claude -p usage. Usage counts against those API credits.
So which is it $300/day is trivial to consume or $200/month is a completely reasonable limit, it can't be both.
Do you even realize how insane your comment is?
"If you aren't donating at least your salary's worth of company money to another company every day, are you even working?"
Used to think exactly like you. That's why I know you all will "get it" eventually. Most companies and orgs are just so far behind the curve.
You might want to put this statement in your AI and ask if it was logically sound
5 replies →
"Used to think exactly like you until I accepted the love of Jesus, our Savior, in my heart."
No AI believer ever gives any concrete examples or evidence of what they’re doing with all the tokens and how it’s objectively helping them make the world a better place. Even for the shareholders (excluding the shareholders of Anthropic, or course), never mind the rest of us.
Why don't you explain it to us then? What are you actually doing with it? What type of products are you working on?
1 reply →
This is a very common pattern with AI psychosis victims (and with crypto and NFT evangelists before). Comments whose haughtiness is matched only by their lack of content.
1 reply →
Apparently both dev's and the AI are vulnerable to the Dunning Kruger effect.
Wouldn't they save an enormous amount of money by getting rid of either you and the token quota, or a bunch of other people to continue paying your salary plus this insane quota?
If you are burning through $2400 a day, you’re just wasting tokens on idiotic tasks.
He's rewriting Bun from Rust to Python now.
How are you able to get to $300/hr productively? (I’m assuming this isn’t fast mode tax).
not hard, massive elo stuff. every decision point needs to think up and implement 25 ideas and then rank them.
I am glad I am not on your team, the amount of slop they have to deal with coming from you must be overwhelming
How? I struggle to use the 1000 Kiro tokens I get a month, and that only costs $20. And I use it more then anyone else on my team. Maybe we're just massively behind?
300 an hour, that's insane
Not really, if you use the most expensive models and you have a large codebase stuffed into the context window
You must be using a really bad harness or just writing very vague prompts. 20 Million tokens is a lot.
[flagged]