Comment by 01100011
11 hours ago
> Deepmind gets to work directly with the TPU team to make custom modifications
You don't think Nvidia has field-service engineers and applications engineers with their big customers? Come on man. There is quite a bit of dialogue between the big players and the chipmaker.
They do, but they need to appease a dozen different teams from a dozen different labs, forcing nvidia to take general approaches and/or dictating approaches and pigeonholing labs into using those methods.
Deepmind can do whatever they want, and get the exact hardware to match it. It's a massive advantage when you can discover a bespoke way of running a filter, and you can get a hardware implementation of it without having to share that with any third parties. If OpenAI takes a new find to Nvidia, everyone else using Nvidia chips gets it too.