Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.
To advertise the maximum possible performance numbers, CPUs and GPUs massively shoot up the power draw for that final few percentage points. If you are worried about actual calculations/$, cutting back on the power will save your energy bill for practically no loss.
When I tested "ECO" mode on my AMD CPU, performance was ~97% of regular mode, and temperatures dropped 5-10C. I think ECO mode drops TDP from something like 100W->65W. Cheaper and cooler to run for basically no cost.
If you are bitcoin mining or selling GPU capacity, you are incredibly conscious of your energy bill, so it only makes sense to optimize the power draw.
Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.
So does llm inference. You're lucky if you hit 40% of the advertised flops.
Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
Why is that true? Can’t you just make more stuff parallel and shrink the ASIC chips accordingly?
No - inability to do so is part of the design of a good cryptographic hash, quite explicitly.
https://en.wikipedia.org/wiki/Avalanche_effect
To advertise the maximum possible performance numbers, CPUs and GPUs massively shoot up the power draw for that final few percentage points. If you are worried about actual calculations/$, cutting back on the power will save your energy bill for practically no loss.
When I tested "ECO" mode on my AMD CPU, performance was ~97% of regular mode, and temperatures dropped 5-10C. I think ECO mode drops TDP from something like 100W->65W. Cheaper and cooler to run for basically no cost.
If you are bitcoin mining or selling GPU capacity, you are incredibly conscious of your energy bill, so it only makes sense to optimize the power draw.
Most were powerlimited to a degree to get the most hash/watt out of them.