Comment by noosphr 6 hours ago So does llm inference. You're lucky if you hit 40% of the advertised flops. 1 comment noosphr Reply fwipsy 2 hours ago Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
fwipsy 2 hours ago Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.
Right, but datacenter GPUs optimized for LLM training/inference would have a bandwidth:compute ratio scaled to that workload.