Comment by andai

2 days ago

So this is cool. I recently asked Opus and Fable to "say hi" in Claude code. The "hi" cost roughly $0.50 for Opus and $1 for Fable, mostly due to the length of the system prompt.

I'm eager to test the new version. If it's 80% shorter now, then saying hello should only cost Fable $0.20. Exciting times.

4 comments

andai

Reply
WASDx  2 days ago

Do you know why they don't just cache the system prompt for everyone? It seems so wasteful not to.

  • andai  2 days ago

    A little bird may have dropped the entire system prompt on my desk.

    They do customisation of the prompt per user and session. (Also apparently per IP address, based on a recent post here...)

    You could still get a lot of savings if you put all the custom stuff at the end but they don't seem to do that. (Well with the old prompt anyway I haven't checked the new one yet.)

    My guess as for why not, is that this is where a lot of their money comes from. So I'm honestly surprised that they removed 80% of the prompt. But I guess my mental model has some gaps in it. Maybe very few of the sessions are short enough for that to matter.