Comment by visarga
1 year ago
I have a strong suspicion that previous generations of TPU were not cost effective for decent AI, explaining Google's reluctance to release complex models. They have had superior translation for years, for example. But scaling it up to the world population? Not possible with TPUs.
It was OpenAI that showed you can actually deploy a large model, like GPT-4, to a large audience. Maybe Google didn't reach the cost efficiency with just internal use that NVIDIA does.
Google used to have superior translation but that hasn't been the case for years now. Based on my experience DeepL (https://www.deepl.com/) is vastly superior, especially for even slightly more niche languages. I'm a native Finnish speaker and I regularly use DeepL to translate Finnish into English in cases where I don't want to do it by hand, and the quality is just way beyond anything Google can do. I've had similar experiences with languages I'm less proficient with but still do understand to an extent, such as French or German
there are several talks out there where Google soft-admits that at least the early gens of TPUs really sucked, e.g.:
https://www.youtube.com/watch?v=nR74lBO5M3s
(note the lede on the TPU is buried pretty deep here)
I suspect it had much more to do with lacking product market fit. They spent 10 years faking demos and dreaming about what they thought AI could do eventually but since it never worked the products never released and so they never expanded. A well optimized TPU will always beat a well optimized GPU on efficiency.
Only because of Nvidia's margins. "Worse but cheaper" is actually great for a company of Googles scale, but it doesn't make for a particularly compelling press release or paper.