Comment by alecco

Grok's latest model is objectively superior to any of the current Google models in the most relevant use cases. I don't like Elon musk but that does not change reality.

And Google came up with the Transformer architecture (2017 "Attention is all you need"). The Attention mechanism they based it on is from Bahdanau, Cho, and Bengio (2014, ICLR 2015). And there were many other self-attention variants by 2017. It was an amazing paper but let's not twist the story and give proper credit.

And not one of the people in that paper are still at Google, AFAIK.

Google has more compute, more data, and had the best 2 labs. And it seems they squandered it all. I'd blame their McKinsey CEO, the board, and management in general. It's a shadow of what it used to be. And it's a shame.