Comment by richard_cory
2 months ago
This is consistently reproducible in completions API with `gpt-5-chat-latest` model:
``` curl 'https://api.openai.com/v1/chat/completions' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer <your-api-key>' \ --data '{ "model": "gpt-5-chat-latest", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "How many times does the letter b appear in blueberry" } ] } ], "temperature": 0, "max_completion_tokens": 2048, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }' ```
hilarious if true, their "gpt-oss-20b" gets it right - however, it still fails on e.g. the German compound word "Dampfschifffahrt" (Dampf-Schiff-Fahrt, steam-ship-journey/ride) because it assumes it's "ff" not "fff"
On the second try gpt-oss-20b gave me "The letter b appears once in the word blueberry."
> because it assumes it's "ff" not "fff"
Funnily enough—and possibly related—this was correct before the German orthography reform of 1996 [https://en.m.wikipedia.org/wiki/German_orthography_reform_of...]
The "gpt-5-chat" model is a non-reasoning model and these struggle because of tokens.