Comment by aesthesia

Ah, I misunderstood what you meant. I was just trying to highlight that in order to answer these types of questions the model needs to memorize the spelling of each token. But you're right that that's all they need to memorize, and algorithms like counting are pretty simple for transformers to implement.