Comment by x1000
2 years ago
There’s a video[1] of Karpathy recounting an email correspondence he had with with Bahdanau. The email explains that the word “Attention” comes from Bengio who, in one of his final reviews of the paper, determined it to be preferable to Bahdanau’s original idea of calling it “RNNSearch”.
"RNNSearch is all you need" probably wouldn't catch on and we'd still be ChatGPT-less.
Worked with PageRank and "map reduce", tho.
Nerds pay attention nevertheless.