Comment by CuriouslyC

The pretraining data isn't getting muddled, the instruction just gets in the lossy middle, and there's less for attention to grab on to. If you repeated the prompt with a hook every turn, it'd stay quite sharp.