Comment by 01100011
13 hours ago
> Deepmind gets to work directly with the TPU team to make custom modifications
You don't think Nvidia has field-service engineers and applications engineers with their big customers? Come on man. There is quite a bit of dialogue between the big players and the chipmaker.
They do, but they need to appease a dozen different teams from a dozen different labs, forcing nvidia to take general approaches and/or dictating approaches and pigeonholing labs into using those methods.
Deepmind can do whatever they want, and get the exact hardware to match it. It's a massive advantage when you can discover a bespoke way of running a filter, and you can get a hardware implementation of it without having to share that with any third parties. If OpenAI takes a new find to Nvidia, everyone else using Nvidia chips gets it too.
This ignores the way it often works: Customer comes to NVDA with a problem and NVDA comes up with a solution. This solution now adds value for every customer.
In your example, if OpenAI makes a massive new find they aren't taking it to NVDA.
Nvidia has the advantage of a broad base of customers that gives it a lot of information on what needs work and it tries to quickly respond to those deficiencies.