Comment by froobius

> Model state is present only in so-far-generated text

Wrong. There's "model state", (I assume you mean hidden layers), not just in the generated text, but also in the initial prompt given to the model. I.e. the model can start its planning from the moment it's given the instruction, without even having predicted a token yet. That's actually what they show in the paper above...

> It is only after the model has found itself in a poetry generating context and has also selected the first line-ending word, that a rhyme scheme "emerges" as a variable

This is an assertion based on flawed reasoning.

(Also, these ideas should really be backed up by evidence and experimentation before asserting them so definitively.)