Comment by dantillberg
7 hours ago
Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.
So does llm inference. You're lucky if you hit 40% of the advertised flops.
Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
Why is that true? Can’t you just make more stuff parallel and shrink the ASIC chips accordingly?
No - inability to do so is part of the design of a good cryptographic hash, quite explicitly.
https://en.wikipedia.org/wiki/Avalanche_effect