Comment by impossiblefork

It's not just competition.

These kinds of things-- cheaper-than-NVIDIA cards that can produce a lot of tokens or run large models cheaply are absolutely necessary to scale text models economically.

Without things like these-- those Euclyd things, those Groq things, etc. no one will be able to offer up big models at prices where people will actually use them, so lack of things like this actually cripples training of big models too.

If the price/token graph is right, this would mean 2.5x more tokens, which presumably means actually using multiple prompts to refine something before producing the output, or to otherwise produce really long non-output sequences during the preparation the output. This also fits really well with the Chinese progress in LLM RL for maths. I suspect all that stuff is totally general and can be applied to non-maths things too.