Comment by bastawhiz

> DeepMind may not be able to just train against all of YouTube just like that

What? Why?

> data quality x data quantity x transformer architecture tweaks x compute cost x talent x time.

Google arguably has the most data (it's search index), the best data (ranked and curated already, along with data sets like books), the cheapest compute (they literally run their own cloud offering and are one of the biggest purchasers of H100s), and the oldest and most mature ML team.