Comment by jfengel

I believe it's reasonably clear that our thought processes generally occur outside of language. We do use language during explicit reasoning, but most thinking occurs heuristically. It's on par with the thinking of animals that don't use language but do complex behavior.

It not clear to me how well that maps onto LLMs. Our wetware predates language, and isn't derived from it. Language is built on top. LLMs are derived from language. I think that means that the intermediate layers are very different from the brain neurons, but I don't know. It's eerie how well the former emulates the latter.