Comment by Isamu

I think you are interpreting “mashup” to mean something like a collage where there’s snippets of a literal piece of this next to a literal piece of that.

But the language model captures not just literal patterns but also patterns of patterns and higher order structures. This is what is mashed up. The model itself doesn’t capture anything beyond what it was trained on but that’s quite enough, it is always a bigger training set than most humans have read on in a lifetime.

This brute force approach is very effective.