Comment by IshKebab

This 100x. I used to work for one of those startups. You need something crazy like a 10x performance advantage to get people to switch from Nvidia to some here-today-gone-tomorrow startup with a custom compiler framework that requires field engineer support to get anything to run.

The outcome is that most of custom chips end up not being sold on the open market; instead their manufacturers run them themselves and sell LLM-as-a-service. E.g. Cerebras, Samba Nova, and you could count Google's TPUs there too.