Comment by famouswaffles

You argue LLMs are Markov chains because the context window is a 'large finite set.' But the physical configuration of the human brain is also a large finite set. We have a finite number of neurons and synaptic states; we do not possess infinite memory or infinite context.

Therefore, by your strict mathematical definition, a human is also a discrete-time Markov chain.

And that is exactly my point: If your definition is broad enough to group N-gram lookup tables, LLMs, and Human Beings into the same category, it is a useless category for this discussion. We are trying to distinguish between simple statistical generators and neural models. Pointing out that they both satisfy the Markov property is technically true, but structurally reductive to the point of absurdity.