In the article, Mitchel mentions how this doesn’t always work. In fact, as someone who’s worked in compiler development, I can say it’s a small miracle when it does work.
Case-in-point, the example in my own post doesn't auto-vectorize with LLVM or GCC at highest optimization levels. Basically, compilers will never auto-vectorize loops with an early loop break afaik.
You would have to give the compiler some help to allow it to auto-vectorize this. Warning, untested: https://godbolt.org/z/b358bMWzG. This unrolls the loop by 16 times. The trick is using `&` instead of `and` so that there's no short-circuiting. All 16 elements are read on each iteration of the loop. This gives the compiler the freedom to replace these reads with a single 16 byte load.
> compilers will never auto-vectorize loops with an early loop break afaik.
You need to let the compiler know that there are at least 4 or 8 elements to process. This may require padding data and/or having a second loop after the main one that processes the remainder <4 or <8 elements.
You start the post with:
> There is an opportunity to use SIMD. SIMD turns those into this:
>
> for (8 byte chunk in bytes) { /* ... */ }
If you actually wrote that loop, there is a good chance the compiler (gcc specifically) will auto-vectorize.
In any case, the more manual SIMD optimizations I have seen require reworking the data altogether, not just processing N elements at a time. For example, instead of packing two 4-vectors into two registers to do a dot product, pack the XXXXs, YYYYs, etc. into 4 vectors and compute 4 dot products for the price of one. That not only requires having 4 vectors to process, but also thinking how exactly they are packed in registers.
I don't know why qurren is downvoted. You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.
Most scalar-to-SIMD conversion requires changing the design of data structures and algorithms to be effective. Compilers are required to exactly reproduce the specified data structures in a deterministic way for obvious reasons.
Even if compilers were clever enough to transform your data structures and algorithms for SIMD (they're not), the data structures are a contract that can't be unilaterally modified.
And -march=native or at least -march=x86-64-v3 or similar, alternatively identifying relevant functions and manually invoking FMV and uarch specialization via target_clones. Plus non-integer code can generally not be autovectorized in normal-math mode since FP is non-commutative.
I have no idea why you're being downvoted. HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers are really good at vectorizing on whatever hardware you're targeting, including SVE.
I mean, utter bullshit. If you "need SIMD" you know exactly the programming pattern to guarantee SIMD from the compiler. And the single and only people who "need SIMD" know these rules. It is only the hobbyist "SIMD is neat" community that upvotes these ridiculous articles.
> HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
What if the “explicit acceleration library” for what you need to do doesn’t exist?
In the article, Mitchel mentions how this doesn’t always work. In fact, as someone who’s worked in compiler development, I can say it’s a small miracle when it does work.
Case-in-point, the example in my own post doesn't auto-vectorize with LLVM or GCC at highest optimization levels. Basically, compilers will never auto-vectorize loops with an early loop break afaik.
You would have to give the compiler some help to allow it to auto-vectorize this. Warning, untested: https://godbolt.org/z/b358bMWzG. This unrolls the loop by 16 times. The trick is using `&` instead of `and` so that there's no short-circuiting. All 16 elements are read on each iteration of the loop. This gives the compiler the freedom to replace these reads with a single 16 byte load.
> compilers will never auto-vectorize loops with an early loop break afaik.
I think this is changing. https://godbolt.org/z/ea1E7dx9v. GCC 14 won't try to vectorize this because of the break statement. But GCC 15 does vectorize it. This got a callout in the "General Improvements" section of the release notes: https://gcc.gnu.org/gcc-15/changes.html
I don't think there's any equivalent in clang/LLVM.
You need to let the compiler know that there are at least 4 or 8 elements to process. This may require padding data and/or having a second loop after the main one that processes the remainder <4 or <8 elements.
You start the post with:
> There is an opportunity to use SIMD. SIMD turns those into this: > > for (8 byte chunk in bytes) { /* ... */ }
If you actually wrote that loop, there is a good chance the compiler (gcc specifically) will auto-vectorize.
In any case, the more manual SIMD optimizations I have seen require reworking the data altogether, not just processing N elements at a time. For example, instead of packing two 4-vectors into two registers to do a dot product, pack the XXXXs, YYYYs, etc. into 4 vectors and compute 4 dot products for the price of one. That not only requires having 4 vectors to process, but also thinking how exactly they are packed in registers.
I don't know why qurren is downvoted. You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.
4 replies →
Most scalar-to-SIMD conversion requires changing the design of data structures and algorithms to be effective. Compilers are required to exactly reproduce the specified data structures in a deterministic way for obvious reasons.
Even if compilers were clever enough to transform your data structures and algorithms for SIMD (they're not), the data structures are a contract that can't be unilaterally modified.
auto-vectorization is not nearly as good as you would hope it to be.
The best SIMD optimizations likely require changing your data format from AoS to SoA.
And -march=native or at least -march=x86-64-v3 or similar, alternatively identifying relevant functions and manually invoking FMV and uarch specialization via target_clones. Plus non-integer code can generally not be autovectorized in normal-math mode since FP is non-commutative.
> The best SIMD optimizations likely require changing your data format from AoS to SoA.
We do have gather load instructions in SIMD instruction sets these days (AVX2 and newer), so AoS vs SoA is not nearly as important as it was once.
Scatter stores are also available but only in newer CPUs.
It remains extremely important. Gather loads are much more expensive than sequential loads, for obvious reasons.
While C++ may be reaching levels of Algol 68, PL/I complexity, with C++26 reflection you can do automatically.
See https://github.com/cern-nextgen/reflmempp
That’s a research project. So no.
The one feature in Jonathan Blow's Jai language I really envy is a a single keyword to switch AoS to SoA and visa-versa at comptime
Didn't he drop this feature years ago?
Either this or you have to do special tricks like pairwise tree reductions and hand-unroll certain portions of loops.
What are AoS and SoA?
Array of Structs and Struct of Arrays https://en.wikipedia.org/wiki/AoS_and_SoA
1 reply →
Array of Structs and Struct of Arrays
Well, then I just prompt Claude and get SIMD without having to learn it /s
I have no idea why you're being downvoted. HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
Every modern language has a vectorization optimizing compiler, and through some fairly straightforward techniques this is automagic. And contrary to the various replies, unless you screwed something up compilers are really good at vectorizing on whatever hardware you're targeting, including SVE.
Compilers are really good but really good is not actually that useful in cases where you need SIMD
I mean, utter bullshit. If you "need SIMD" you know exactly the programming pattern to guarantee SIMD from the compiler. And the single and only people who "need SIMD" know these rules. It is only the hobbyist "SIMD is neat" community that upvotes these ridiculous articles.
1 reply →
> HN has a fetish for SIMD, but if you are hand-rolling SIMD and you aren't writing an explicit acceleration library, you're doing it wrong. Like, 100% of the time.
What if the “explicit acceleration library” for what you need to do doesn’t exist?
Or it exists and is not optimal for your use case?