Comment by aurareturn

I don't think ASICs is the long term answer here at the local LLM level. I think GPUs will still be.

If you can have only one AI processor in your laptop (because they're big and expensive), it's going to be a GPU. This AI processor needs to inference LLMs, audio processing, image generation, video generation, etc. This is on top of normal graphics processing requirements such as video games, playing videos, decoding, encoding, etc.

At the enterprise level, I can see some ASICs working once the market fully matures and improvements in architectures slow down drastically while demand for inference increases drastically. How far are we from this world? Maybe 5-10 years? It seems like model architectures are still changing rapidly and labs want fast experimentation that programmable GPUs offer.

GPUs will still dominate in general - just like how CPUs still dominate despite ASICs.