Comment by brookst
6 months ago
State typically means between interactions. By this definition a simple for loop has “hidden state” in the counter.
6 months ago
State typically means between interactions. By this definition a simple for loop has “hidden state” in the counter.
Hidden layer is a term of art in machine learning / neural network research. See https://en.wikipedia.org/wiki/Hidden_layer . Somehow this term mutated into "hidden state", which in informal contexts does seem to be used quite often the way the grandparent comment used it.
It makes sense in LLM context because the processing of these is time-sequential in LLM's internal time.