A lot of companies that serve a single family of LLMs seem to prefer nvidia though. Why is that?
It's not just good drivers, which is what moats them for games and ML. It's a multi-decade work of making chips that are nice to program for and software infrastructure around them.
Apple and Google have excelent chips, yet they needed to invest a lot in long-tail software projects to make those chips do actual premium work. Still not state of the art for serving LLMs (although Google is strong in that, mostly because it piggybacked on previous chip-related software work for phones and so on).
What will people use to write for Jalapeño matters.
Nvidia has multi-decade heritage. Apple spent almost a decade in MLX. Snapdragon failed partly here. OpenAI announced nothing regarding to that, so this big moat that multiple companies have (nvidia the most prominent) is nil for them.
A lot of companies that serve a single family of LLMs seem to prefer nvidia though. Why is that?
It's not just good drivers, which is what moats them for games and ML. It's a multi-decade work of making chips that are nice to program for and software infrastructure around them.
Apple and Google have excelent chips, yet they needed to invest a lot in long-tail software projects to make those chips do actual premium work. Still not state of the art for serving LLMs (although Google is strong in that, mostly because it piggybacked on previous chip-related software work for phones and so on).
> A lot of companies that serve a single family of LLMs seem to prefer nvidia though. Why is that?
If you write your tools for CUDA, you’re going to prefer hardware the runs CUSA.
How is there anything more to it than this?
Cool. That's it.
What will people use to write for Jalapeño matters.
Nvidia has multi-decade heritage. Apple spent almost a decade in MLX. Snapdragon failed partly here. OpenAI announced nothing regarding to that, so this big moat that multiple companies have (nvidia the most prominent) is nil for them.