Comment by orwin

Transformers vectorize words in n dimensions before processing them, that's why they're very good at translation (basically they vectorize the English sentence, then devectorize in Spanish or whatever). Once the sentence is processed, 'blueberry' is a vector that occupy basically the same place as other berries, and probably other. The GPT will make a probabilistic choice (probably artificially weighted towards strawberry),and it isn't always blueberry.