Comment by abetusk
6 hours ago
I'm not sure I had a well formed question. I saw the post title, didn't get a clarification in the article and started searching.
The entropy rate seems like a pretty natural definition of "entropy of a Markov chain", no? It's not exactly this but it's similar to "start at state i, end on state j (maybe in n steps?), what is the number of bits I need to send over the wire to tell you what path was taken".
What does the entropy of the raw stationary distribution give you? Is the entropy rate related to the entropy of the stationary distribution (the thermodynamic entropy?)?
I was probably projecting. I like both path and equilibrium entropies, but my assumption I guess was that when people hear entropy in physics they're usually thinking of the equilibrium-defining entropy. That is, the value that is maximized as the system relaxes to equilibrium, the vanilla thermodynamic entropy.