Comment by willy_k
6 hours ago
It took 6 years to get from Attention Is All You Need to a consumer product. Up until ChatGPT’s release, LLMs were obscure and really were just fancier autocomplete. Deepmind was largely a speculative research division until Google decided to play catch up, and even that took a bit of time as they figured out how to proceed in a way that wouldn’t cannabalize search. So in short they didn’t realize the potential. Thats my somewhat naive take.
Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.
and Google Translate: https://arxiv.org/abs/1804.09849