Comment by harrall
10 hours ago
Google invented the AI architecture that Anthropic and OpenAI based their entire companies on? Based off years of research at Google.
Of course they should have to fight with the inventors of the technology they’re using.
> Google invented the AI architecture that Anthropic and OpenAI based their entire companies on
Source?
Unless you don’t think Attention Is All You Need?
https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
"Attention Is All You Need" was a paper by a bunch of Google researchers
Still, attributing that progress to "years of research at Google" alone is simplifying the facts to the point of being just plain wrong. That kind of research was always very much in the open and cooperative, with deep levels of standing-on-shoulders.
Attention e.g. was developed by Dzmitry Bahdanau et al. (those being Kyunghyun Cho and Yoshua Bengio) in 2014 while interning at the University of Montreal.
The insight of the paper you point to was that with attention you could dispense of the RNN that attention was initially developed to support.
1 reply →