Comment by gscott

6 days ago

It seems FPGA can now do things for LLM's so there might be some future in that

8 comments

gscott

Its never going to be as efficient as ASIC and the LLM market is definitely big enough for ASICs to be viable.

imtringued 6 days ago
I hear this a lot, but in my experience this isn't true at all.
A Versal AI Edge FPGA has a theoretical performance of 0.7TFLOPs just from the DSPs alone, while consuming less power than a Raspberry Pi 5 and this is ignoring the AI Engines, which are exactly the ASICs that you are talking about. They are more power efficient than GPUs, because they don't need to pretend to run multiple threads each with their own register files or hide memory latency by swapping warps. Their 2D NOC plus cascaded connections allow them to have a really high internal memory bandwidth in-between the tiles at low power.
What they are missing is processing in memory, specifically LPDDR-PIM for GEMV acceleration. The memory controllers simply can't deliver a memory bandwidth that is competitive with what Nvidia has and I'm talking about boards like Jetson Orin here.
- _hark 5 days ago
  
  If FPGAs are competitive on perf/watt, why aren't they more widespread (other than crap software tooling)?
  Honestly I've asked different hardware researchers this question and they all seem to give different answers.
  
  1 reply →
- wmf 5 days ago
  
  Now compare that FPGA to Groq, SambaNova, or Cerebras. The ASICs are more efficient.
  
  1 reply →
lesuorac 6 days ago

This is past tense no?
There's been neural processing chips since before LLM craze [1].
[1]: https://en.wikipedia.org/wiki/Neural_processing_unit#History

If LLM can leverage on the new efficient attention mechanism based the FFT architecture discovered by Google then FPGA can be the new hot stuff [1]:

[1] The FFT Strikes Back: An Efficient Alternative to Self-Attention (168 comments):