← Back to context

Comment by minraws

4 hours ago

I don't want to sound rude but even at cache hit you are burning almost input * 0.10 * context amount of tokens multiple times.

If it happens 10 times you have already accepted the same cost as the entire session fresh prefill.

People on HN are either too rich or wayyy too rich

But you also burn the exact same number of tokens (actually more) every time the model makes a tool call. It's an agent, it makes a lot of tool calls.

  • This is the right perspective. It costs as much as one additional tool call in the conversation. It’s a bit weird to do it on a timer, which inflates a bit over “just a tool call”, but not by much?