Comment by semiquaver

It’s not sampling anything though. At each inference step the fundamental output of the process can be interpreted as a probability for each possible token how likely it is. Sampling would imply otherwise but the probabilities are the output.