Comment by HeavenFox
3 hours ago
If they are literally sending the entire transcript and asking for a summary, wouldn't almost all of the tokens be cache hit, and therefore fairly cheap?
3 hours ago
If they are literally sending the entire transcript and asking for a summary, wouldn't almost all of the tokens be cache hit, and therefore fairly cheap?
I don't want to sound rude but even at cache hit you are burning almost input * 0.10 * context amount of tokens multiple times.
If it happens 10 times you have already accepted the same cost as the entire session fresh prefill.
People on HN are either too rich or wayyy too rich
But you also burn the exact same number of tokens (actually more) every time the model makes a tool call. It's an agent, it makes a lot of tool calls.
Who pays for their own tokens at full price?
All Claude Enterprise customers.
given that the subsidized plans from all companies are getting worse by the hour it's probably a smart thing to consider, at least.
Isn’t it 0.2?
It's 0.1 I think brain fart I typed the rest correctly though after 10 summaries you still burn the same amount of tokens
A cheap operation adds up to a large cost if it's done enough times. TFA even talks about caching.
Do you disagree with the numbers and conclusion in the -pretty obviously bot-generated- Fine Article?
Didn't they change the cache TTL to 5 minutes earlier this year? So depending on exact timing it could potentially be invalidated between the updates.
If you're using the API, I think. It's an hour when using Claude Code.
It's an hour. It'll tell you the expiry time on the status line now but that information is also available in /usage.
https://github.com/anthropics/claude-code/issues/46829
Looks like what I remembered was a bug.
Unless they prepend something like, "summarise this, "
But they definitely wouldn’t do that.