Comment by olavgg
13 hours ago
Its easy when you spawn a couple of subagents in a homelab with a rack with 5+ servers.
This is how one of my claude code session looks like right now, one prompt from yesterday:
◯ lean-tracks ▰▰▰▰▰▰▰▰▰▰▰▱▱▱▱▱▱▱▱▱ 7/12 · 18h35m · ↓ 3.1m tokens
◯ lean-tracks ▰▰▰▰▰▰▰▰▰▰▰▰▱▱▱▱▱▱▱▱ 6/10 · 17h47m · ↓ 2.6m tokens
◯ lean-tracks ▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱ 0/1 · 4h49m · ↓ 275.9k tokens
◯ lean-tracks ▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱ 0/1 · 4h30m · ↓ 395.0k tokens
◯ lean-tracks ▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱▱ 0/1 · 3h50m · ↓ 490.3k tokens
Not sure if this will help you but it helped me. I was using subagents and workflows for lots of long running tasks, and part of that work included running tests. If tests took longer than 5 minutes, the cache expired, and the whole thing was then being processed as uncached input.
I created a skill to better manage agents, including a way to keep the caches valid, and it has decreased my usage considerably. I now struggle to use my 100% each week and I hit claude hard. I have a reset expiring in 5 days and I'm trying to burn down the tokens (68% with 3d 8h left) quickly so I can use my reset before it expires.
Token efficiency just means understanding the way the system works. Best person to help with that... my old friend claude.
After I stopped letting Claude run tests, my token usage went from "gone by Wednesday" to "what can I automate this Saturday so my tokens won't go to waste".
Thanks this is very helpful, i didn't realize that. But not sure what to do since running tests is how Claude often catches mistakes and iterates. If i run them myself, the cache will still expire.
However where i think you have a point is that it tends to run tests quite casually where a reviewer would have been enough to catch mistakes.
IMHO, the real problem is the 5min cache way too small. They should up it to 1 hour.
What is the reason for this? Surely running tests by itself isn't necessarily token-expensive? And it provides cheap feedback for the agent: to see whether they're breaking things.
6 replies →
How do you do that?
1 reply →
Yeah, I should see if I can create a similar skill. The tests do spend some time to complete.
Claude: Confirmed: 195M of the 212M cache tokens (92%) come from full re-writes after 5+ minute gaps, accounting for 77% of relative cost. Now I'll check the existing wrappers so the skill builds on them.
Yesterday in my CC there was an update that added a feature to auto compact immediately prior to the cache being invalidated.
And I immediately turned it off because that just invites utter churn and stupidity. Part of that is just CC though, their compaction is miles behind Codex.
Some to think about with both Claude and skills.
Can you invert things. What’s deterministic and what needs “thinking”?
Can you create a script that runs things chaining together the deterministic parts with calling out to Claude for glue that needs “thinking”.
Don't suppose you would be happy to share the skill? Or is it fairly unique to your work flow?
I think the tests problem is really having an impact for me. Hadn't considered it before so thanks for flagging.
How do you keep the caches valid?
Not OP, but I typically ask it to report status of whatever is running; it can run a background task that reports to it every N minutes. With Claude Code in VS Code and a Pro subscription, it seems like the one-hour cache is the default, making it an issue less often.
Ask Claude, it's pretty wild to see.
> including a way to keep the caches valid
Maybe doing bogus small requests to prevent the cache from expiring?
I mean in the API you can set the cache expiration time right? But I guess claude code can't do this
> my old friend claude.
Awfully new old friend.
Google is the ex, Altavista is the high school sweetheart who tragically passed at a young age.
I have some fairly large personal software projects and I still don't understand what useful work people are getting out of burning all these tokens on unattended agents.
What exactly are they doing for you that's giving you $200 of value in such a short timeframe?
As a dev who hasn't used LLMs as an embedded part of their work, and uncertain about it all, i'd be curious to know:
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
For what I am currently doing, years, if not a decade or maybe even two. Claude has already been running for 3 months constantly working on this project. I've been trying to do this several times before, but the scope outgrew my available time/life, too big for one person. But now I can actually build a production release by just setting the direction and let Claude to all the wiring, plumbing, tests etc.
I do now add all the major features I want, and write a task list where I add a few more minor tasks each week that just come to my mind. I need to write them down before I forget. I often ask Claude to implement several different implementations/algorithms and compare them. There are plenty of trade-offs, which usually end up in building configuration profiles, so the application can work optimally in multiple environments.
The specification is actually really large, and has grown over time as I've been learning and new ideas has come to my mind. I do not review/audit the code, it's massive would take me months, no chance! I do however use code inspection/audit tools like TSAN, ASAN and more. I will also spend time on full code reviews with the coming Fable 5.5 when a release is about ready. I do however review behavior together with Claude, and this is also where the specification grow, because this and that should be possible, but not this and that. User stories also helps.
Thanks, appreciate the reply.
Keep doing what you're doing, this guy clearly is just revving his engine because he likes the way it sounds and can afford the gasoline. I wouldnt be asking people like this for advise, they're completely off the rails.
There's no reason to blow through tokens this way, there's a 99% chance nobody uses the output (except maybe themselves).
Yes, but what was the task that they were working on? This tells nothing.
That’s the “how” we’re wondering about.
[flagged]
You're doing this wrong.