Comment by MangoCoffee
4 hours ago
Nvidia is trying to increase its customer base. Look at the Mag 7, Amazon, Google, Microsoft, and even Meta are all working on their own inference chips. I don't think they can completely ditch Nvidia for LLM training, but they can make their own chips for inference. I believe that's also why OpenAI made its own.
No one wants to pay the Nvidia tax
If you make your own chip don't you need to make your own software stack too. Which is what I thought kept everyone using Nvidia.
> Which is what I thought kept everyone using Nvidia.
Once an organization is big enough to have AI-platform or AI-infra teams, you stop being hung-up on details like CUDA; resources will be invested to make whatever abstractions the AI researchers and engineers rely on (e.g. Torch) will be made to work, and work well.