Comment by argee
20 hours ago
> The difference in an attention layer is that the query is matched to each key using a dot product, and rather than getting a single value back you get a weighted sum of values.
I understand you meant to compare it to dicts, but this is more like (the only?) similarity to outer product memory. Worth calling out since GP is confused about this connection specifically.
Yes, the main problem with this whole way of thinking about transformers and self-attention is that when you look at how they are actually implemented (i.e. what Q, K, and V are applied to at each layer, and consider the loss function during training), then the analogy to memory quickly starts to look tenuous and far-removed from what is actually happening.
Mainly, since self-attention layers operate on deeply-stacked representations of the outputs of previous layers, and since training has a loss function that doesn't make the updates work in the way the article says, the analogy to memory and lookup tables breaks down quite strongly. IMO, thinking of Q, K and V as memory or key-value lookup is thus not actually very helpful for understanding what these layers are really doing or why they work. Others agree (e.g. https://arxiv.org/html/2410.18613v1), and because much of what works in DL is usually more about matrix conditioning and keeping gradients well-behaved generally, rather than what the actual bare final math / weights in isolation from training would tend to suggest. But it does give a great intuition for why a trained transformer can have the capacity it does.
Definitely! It's more of an intuition but doesn't explain what vector gets put in the weights and the analysis doesn't involve the learning dynamics at all. I link some papers at the end that do think about how memory relates to the dataset. But the architecture having this storing+lookup property probably is important in helping transformers remember so much information.