Comment by johnecheck
2 months ago
That proves nothing. The fact that Mississippi has 4 "s" is far more likely to be in the training data than the fact that blueberry has 2 "b"s.
And now that fact is going to be in the data for the next round of training. We'll need to need to try some other words on the next model.
It does the same thing with a bunch of different words like "committee", "disestablishmentarianism", "dog", "Anaxagoras", and a string I typed by mashing the keyboard, "jwfekduadasjeudapu". It seems fairly general and to perform pretty reliably.
(Sometimes the trace is noisier, especially in quants other than the original.)
This task is pretty simple and I think can be solved easily with the same kind of statistical pattern matching these models use to write other text.