Comment by Decabytes

> Why has it taken this long to catch on?

Intel didn't put SIMD in their chips speculatively; there was a lucrative market. HPC presumably. Maybe SIGINT.

There have always been SIMD accelerated libraries, but I assume that you're talking about more widespread hand-coded SIMD application code (either in assembly or using compiler intrinsics.) For audio DSP this caught on when SSE2 launched around 2000-2002[0]. All things being equal, the user being able to run twice as many of your plugin as your competitor's is a real commercial advantage (being super-slow compared to your competitors is also not a great look.)

Not all applications have such forces at play. For most of the intervening period computers were getting faster every generation anyway, so users were seeing performance improvements when they upgraded their hardware, with no developer effort. For me, adding features delivered more value to users than making the existing product run faster.

A few things that factored into my personal limited use of SIMD over that time frame (even though I did write some inline asm): overhead of maintaining compatibility with multiple CPU generations. i.e. AMD and Intel were not in lockstep (still aren't, see AVX512); my users' hardware was all over the shop ranging from latest and greatest to older than you would imagine; I wanted good OS backward compatibility; autovectorisation was supposed to be just around the corner (honestly it's not bad if you limit the compiler to a recent instruction set). All of this could have been addressed, but features were a higher priority.

In addition, that period was the era of "write once run everywhere." Portable code was desirable. Having inline asm was seen as dirty, using instructions that only ran on specific makes and models of CPU more so. SIMD was alien to the mainstream developer zeitgeist.

In recent years the calculus has changed: CPU clock speeds are stagnant, most if not all application processors have SIMD units. More people understand that most CPU compute is in the SIMD units. We seem to have reached a critical mass of open SIMD algorithm development in less niche domains (e.g. simdjson). But SIMD code is still not write-once-run-everywhere.

[0] https://en.wikipedia.org/wiki/Streaming_SIMD_Extensions

Remember the part of the article where he talks about a shocking number of people didnt have the version of AVX he wanted. This is a big issue because SIMD is writing ASM for the most part not C code so new architecture you have problems . RVV riscv SIMD has an interesting way to adapt to different CPU tiering and is a rather complete SIMD architecture but most SIMD architectures have small holes which can be mildly annoying

Simd is by far the most efficient cycles wise paradigm but its also brittle. When the shape of data is odd which is common enough you have to handle it properly which is sometimes a distraction from what people actually care about. Also GPUs exist, GPUs despite being way less power efficient are not brittle in this same way they have issues if memory is difficult but most problems work nicely on GPUs so people just do that because its easier and the tiering problem is less a big deal because GPU process instructions different than SIMD and batching and what not is left to hardware not the programmer

The average programmer isn’t falling into pits of success with auto vectorization.

And you’re using languages like Python or Lua and not using a dependency that uses it for you, forget about it.

https://en.wikipedia.org/wiki/Single_instruction,_multiple_d...

Looking at the Wikipedia article - I wonder if it is because SIMD and SIMT are enabled by hardware first

LLMs?