Comment by visarga

I think you are missing the conditional, contextual nature of language models. They mix things in coherent ways, they adapt to the request. Google doesn't create new things when they don't exist, and the pre-written code examples on the internet will never adapt to your needs.

But I agree with you that everything they do seems intelligent because 'intelligence' was in the training data. Not much different from us, if you raise a human removed from society (take his intelligent training data away) he will not accomplish almost anything on his own.