Comment by jeffbee
16 hours ago
Urs used to talk (internally) about not publishing "industry-enabling papers" which is why most Google infrastructure papers were describing something that had already been turned off, or was already in the process of being replaced by the next system (GFS, Vitess, etc). The things that did get published were either considered not key advantages, that other companies simply cannot do, things that other companies wouldn't bother doing, or experiments that never worked at all. There were exceptions of course. But it led to a public perception of the Google stack involving mostly technologies that were long dead or were never adopted.
"Attention Is All You Need" was a very very different thing and I also wonder if they are glad they published it. But I imagine if they hadn't, the motivation for researchers to leave Google would have been even larger.
> I also wonder if they are glad they published it
https://youtu.be/ue9MWfvMylE
Jeff Dean is asked this question by Geoffrey Hinton at 37:35 - might worth watching. Overall an interesting video.
Link with time code: https://youtu.be/ue9MWfvMylE?t=2261
So Google allowed publishing the Attention paper because they didn't understand its value.
They patented it. When the dumb money stops sloshing around, we'll start to see the fallout from that.