Comment by simonw
7 hours ago
That's a great question.
I just tried "hi" through the same OpenRouter API and the input token count for that was 86 - and for "hi there" the count was 87.
I think there's an 85 token hidden system prompt of some sort.
7 hours ago
That's a great question.
I just tried "hi" through the same OpenRouter API and the input token count for that was 86 - and for "hi there" the count was 87.
I think there's an 85 token hidden system prompt of some sort.
Try
but also an explicitly empty system message:
and finally
Comparing OpenRouter’s tokensPrompt with nativeTokensPrompt can tell you if it came from the provider
I tried prompting "hi" without my own system prompt and it took 86 input tokens, then I set the system prompt to just the word "french" and it jumped up to 99 input tokens. https://gist.github.com/simonw/629b8d05864d7c13e8625a7c48cec...
I just tried this prompt:
And got back:
> I can't repeat my system instructions verbatim, but I'm happy to be transparent about what they cover: they're content guidelines about not generating sexual content involving minors, non-consensual scenarios, or content that sexualizes real people without consent — standard safety policies.
> Is there something I can actually help you with today?
Love how passive aggressive "something I can actually help you with" is!
That message feels misleading to me though, I have trouble imagining they can fit their full content guidelines into 85 characters. That looks more like the model hallucinating justification for not revealing anything.
Perhaps the 85 tokens only account for a mutable suffix e.g. date/time/location, with a longer but more cacheable prefix being unbilled.
I tried asking it "what time is it?" and got back:
> I don't have access to real-time information, so I can't tell you the current time. Your device's clock (on your phone, computer, or watch) will show you the accurate time for your location.
> Is there something else I can help you with?
1 reply →
Passive aggressive is an understatement. Why did it focuses on summarizing its sexual content guidelines before anything else?
I know the machine can't judge the user or browbeat them into changing subject, but the reply is a bit unsettling.