Comment by charles_irl

4 days ago

100% -- there's basically no substitue for benchmarking! I find the empiricism kind of comforting, coming from a research science background.

IIUC, even CuBLAS basically just uses a bunch of heuristics that are mostly derived from benchmarking to decide with kernels to use.