Comment by RossBencina
2 days ago
> Why has it taken this long to catch on?
Intel didn't put SIMD in their chips speculatively; there was a lucrative market. HPC presumably. Maybe SIGINT.
There have always been SIMD accelerated libraries, but I assume that you're talking about more widespread hand-coded SIMD application code (either in assembly or using compiler intrinsics.) For audio DSP this caught on when SSE2 launched around 2000-2002[0]. All things being equal, the user being able to run twice as many of your plugin as your competitor's is a real commercial advantage (being super-slow compared to your competitors is also not a great look.)
Not all applications have such forces at play. For most of the intervening period computers were getting faster every generation anyway, so users were seeing performance improvements when they upgraded their hardware, with no developer effort. For me, adding features delivered more value to users than making the existing product run faster.
A few things that factored into my personal limited use of SIMD over that time frame (even though I did write some inline asm): overhead of maintaining compatibility with multiple CPU generations. i.e. AMD and Intel were not in lockstep (still aren't, see AVX512); my users' hardware was all over the shop ranging from latest and greatest to older than you would imagine; I wanted good OS backward compatibility; autovectorisation was supposed to be just around the corner (honestly it's not bad if you limit the compiler to a recent instruction set). All of this could have been addressed, but features were a higher priority.
In addition, that period was the era of "write once run everywhere." Portable code was desirable. Having inline asm was seen as dirty, using instructions that only ran on specific makes and models of CPU more so. SIMD was alien to the mainstream developer zeitgeist.
In recent years the calculus has changed: CPU clock speeds are stagnant, most if not all application processors have SIMD units. More people understand that most CPU compute is in the SIMD units. We seem to have reached a critical mass of open SIMD algorithm development in less niche domains (e.g. simdjson). But SIMD code is still not write-once-run-everywhere.
> But SIMD code is still not write-once-run-everywhere.
Modern, portable SIMD libraries are making it increasingly close to that. Highway is my personal favourite, thanks in part for its support for runtime dispatch.
(Disclaimer: I work at Google, but not on Highway, and am writing in my personal capacity.)