Comment by vatsachak

Great article. My favorite fact about "transformers as a hopfield network" is that transformers are so good at memorizing that they only need one step to find a suitable memory as opposed to multiple steps for a vanilla hopfield network