Comment by cookiengineer

It's actually an effect that happens in the (re-)alignment process due to harmonic properties of the positional encoding in the attention matrix.

(I recommend reading and implementing the Attention is all you need paper. By hand. Otherwise you won't learn anything from it.)