Still, attributing that progress to "years of research at Google" alone is simplifying the facts to the point of being just plain wrong. That kind of research was always very much in the open and cooperative, with deep levels of standing-on-shoulders.
Attention e.g. was developed by Dzmitry Bahdanau et al. (those being Kyunghyun Cho and Yoshua Bengio) in 2014 while interning at the University of Montreal.
The insight of the paper you point to was that with attention you could dispense of the RNN that attention was initially developed to support.
Still, attributing that progress to "years of research at Google" alone is simplifying the facts to the point of being just plain wrong. That kind of research was always very much in the open and cooperative, with deep levels of standing-on-shoulders.
Attention e.g. was developed by Dzmitry Bahdanau et al. (those being Kyunghyun Cho and Yoshua Bengio) in 2014 while interning at the University of Montreal.
The insight of the paper you point to was that with attention you could dispense of the RNN that attention was initially developed to support.
Context: https://news.ycombinator.com/item?id=42310213