Comment by dragonwriter
10 days ago
> Both LLMs and n-gram models satisfy the markov property, and you could in principle go through and compute explicit transition matrices (something on the size of vocab_size*context_size I think).
Isn’t it actually (vocab_size)^(context_size)?
Yes, you're right. I typed "**" (exponentiation) but HN ate the second star since I forgot to escape.