Comment by albertzeyer
2 years ago
I would say the opposite. This paper was a very easy read, totally clear from the first reading what it is about, etc.
The background matters. Attention was already very well known in the community (machine translation), so nothing new for this paper, and it was written for such an audience which already knows these basics concepts like attention.
If you want to learn about attention, read some of the actual background papers which introduced it.
Can you link some of those papers?
I am not sure if this was the very first paper talking about attention, but https://arxiv.org/abs/1409.0473 from 2014 is a famous one.
What is still missing there, is the idea of self-attention.