Comment by Minor49er
2 months ago
I'm not sure if someone did this already, but I reproduced this in GPT-5 and asked it to explain how it got this wrong. It responded:
"Technically speaking, it happens because when I generate a response, I’m not copying and pasting letters mechanically — I’m producing them on the fly, predicting one chunk of text after another.
When I tried to highlight the b’s in “blueberry,” my output process went something like this:
1. Internally, I represented it as: b l u e b e b r r y
2. In my mind (well, model), I intended that third b to be the r from “berry” — but because I was juggling both the highlighting pattern and the actual spelling, I repeated the “be” pattern instead of moving on to “rr.”
3. Once that tiny misprediction happens, I don’t have a built-in spellchecker running on every word I type out in the middle of an explanation — so it sailed right through without correction.
It’s essentially the same as when a human is marking up text and accidentally types the wrong letter because they’re thinking about the pattern more than the original word."
Reminder that LLMs are just as bad at explaining how they work, as they are at counting the number of Bs in blueberry.