Comment by simonw
12 hours ago
These system prompts don't affect the API, they are for the Claude consumer chat products. We aren't charged extra for them.
They're also prefix cached, so the cost to Anthropic and performance hit is greatly reduced.
12 hours ago
These system prompts don't affect the API, they are for the Claude consumer chat products. We aren't charged extra for them.
They're also prefix cached, so the cost to Anthropic and performance hit is greatly reduced.
They have a {{currentDateTime}} in the prompt which is interesting for something that gets prefix cached. Hopefully they are handling that properly :)
So the people using the Claude consumer chat products pay for them via usage...
That's not any better. It's actually worse.
I don't understand.
System prompts are part of the software that customers pay to access.
Complaining about that is a bit like complaining that your Netflix subscription includes paying to execute the compiled code that Netflix wrote that serves you video streams from their servers.
Actually there is a difference: If Anthropic deleted a large chunk of that system prompt I guess you might get like a 1% increase in how much Opus 5 you can use via their chat allowance for your paid subscription.
Is that really something worth being frustrated by?
Cached.
they are the first part of the input and it contains no user dependent variables, so the model is in a known state that it can reuse across all users, it does not need to recompute all that inference
Unless they are using a linear architecture, the compute cost still scales O(n²) for n tokens, and nemory cost scales O(n).
2 replies →