Comment by Oras
17 hours ago
They just announced their new chip, and they are the ones created transformers yet investing this amount in a competitor?
I don’t know what to make of it
17 hours ago
They just announced their new chip, and they are the ones created transformers yet investing this amount in a competitor?
I don’t know what to make of it
I wonder if Google regrets publishing that article on transformers.
Urs used to talk (internally) about not publishing "industry-enabling papers" which is why most Google infrastructure papers were describing something that had already been turned off, or was already in the process of being replaced by the next system (GFS, Vitess, etc). The things that did get published were either considered not key advantages, that other companies simply cannot do, things that other companies wouldn't bother doing, or experiments that never worked at all. There were exceptions of course. But it led to a public perception of the Google stack involving mostly technologies that were long dead or were never adopted.
"Attention Is All You Need" was a very very different thing and I also wonder if they are glad they published it. But I imagine if they hadn't, the motivation for researchers to leave Google would have been even larger.
> I also wonder if they are glad they published it
https://youtu.be/ue9MWfvMylE
Jeff Dean is asked this question by Geoffrey Hinton at 37:35 - might worth watching. Overall an interesting video.
1 reply →
So Google allowed publishing the Attention paper because they didn't understand its value.
1 reply →
Why do you think Google considers Anthropic a competitor?
It makes every bit as much sense as investing in Snap while still operating their own social network product. Seems to have worked out fine (for Google, not Snap).
FWIW I’d buy SNAP now that they are at rock bottom
Given that anthropic is probably paying it all back to them in compute bills, they may not be giving them anything.
hedge your bets, I know I would