Comment by stochastic_monk

8 years ago

I’ve found template metaprogramming to be an efficient, generic way to generate SIMD-accelerated code. (See [0].) By giving the type functions associated with each operation, the same interface can support 128, 256, or 512-bit SIMD, depending on hardware.

It’s C++, so while it’s outside of the Rust ecosystem, it’s still a workable solution I’ve used in a half dozen projects.

[0] https://github.com/dnbaker/vec

6 comments

stochastic_monk

Reply
eslaught  8 years ago

I don't know if you've ever seen Kokkos[1], but that's one of the big C++ frameworks for taking parallel loops and using templates to generate code for SIMD and multiple cores. They also support GPU code generation to some degree. The main issue is compile time, which gets pretty bad when you start using it in non-toy applications.

I can't find the reference right now, but there have been some attempts to augment C++ compilers to understand the semantics of the library directly, basically treating the template metaprogramming as a DSL instead of general-purpose. This can be speed up the compile times dramatically, but of course you lose generality and the compiler has to be customized to understand every library it wants to optimize. Overall it seems like there is more research to be done on doing this in a safe way without paying through the nose with compile time.

[1]: https://github.com/kokkos/kokkos

grandinj  8 years ago

That doesn't give you run-time dispatch based on available hardware capabilities. So you either need virtual functions (and take the associated perf hit), or some other solution like the linker tricks mentions, or cached JIT, or....

  • stochastic_monk  8 years ago

    It’s not runtime, it’s compile-time. That means it needs to be separately compiled for each hardware backend.

    • grandinj  8 years ago

      How do you combine the separately compiled libraries into a single deliverable library/executable?

      Because that is what the original article is aiming for.

      1 reply →

  • gameswithgo  8 years ago

    it can, you generate all desired version of simd-ified function at compile time, then write a function to select the right one at runtime.

    that is what the Rust libs are doing, just using traits or macros instead of templates.