Comment by srean

I covered hidden Markov Models briefly in my comment.

It depends on whether the state is visible in the observations or not. Hidden or not is an orthogonal axis of variation compared to the other variations mentioned in the comment.

In a non-hidden model there is no ambiguity or uncertainty about what the current state is.