Comment by m_mueller
8 years ago
As someone coming from CUDA I think you‘re on the right path. CUDA unifies multicore and vectorization parallelims. Crucial here is hardware support for branching (including early returns) even if it degrades performance. This allows a CUDA kernel being programmed in a programmer friendly way that is often already fairly performant, and the actual mapping to hardware is deferred to kernel invocation where it‘s straightforward to specialize. I think this concept is something that CPU (programming) could learn from GPU. Why not adopt the concept of kernels?
OpenCL on CPU is exactly this and does very well. Last real world test I did, an 8 core Haswell and a GTX 970 were similar in performance, for the kernel I was running. Caveats apply of course but it was refreshing to have a single programming model.
I did some evaluation of (Linux based) OpenCL implementations two years ago (but I don't think anything changed significantly since then).
I had a few takeaways:
- OpenCL on GPUs and CPUs have little to do with each other (in terms of performance characteristics) and if you tune well for one of them, the other one will suffer.
- Vectorization of work items doesn't really work well unless your kernel is so simple that normal compiler auto vectorization with a loop would have probably worked just as well if not better.
- Nvidia intentionally makes OpenCL a second class citizen vs. CUDA. I had nearly identical (simple) kernels running on both platforms and only the CUDA one managed to saturate memory throughput.
- The whole ecosystem is mostly more effort than it's worth. Portability between different OpenCL implementations is a gamble, some will even silently compute invalid results (I'm looking at you Intel...). I had kernel hangs with both Nvidia and AMD.
> if you tune well for one of them, the other one will suffer.
This is one of the things that bothers me the most about OpenCL. It attempts to offer this uniform abstraction over a generic compute accelerator, which can be CPU vector extensions, GPUs, or FPGAs, but these things are different enough that you have to develop for a specific type if you want reasonable performance. So you get none of the benefits of a accelerator specific abstraction while still writing accelerator specific code.
There is a real cost to a generic abstraction, and distinct languages/platforms would in my mind be better than different "dialects" of the same language/platform that pretend to be compatible but really aren't.
I like that CUDA is very clearly designed to run only on GPUs - it provides a clarity that OpenCL lacks.
The Intel OpenCL driver vectorized work items really well, better than the same code provided to ICC as C. It was still more fragile than CUDA.
The other points are spot on and I would add that debugging code in OpenCL is a bad experience.
I agree that this space needs to be explored more. ISPC is basically this, and it had some great ideas, it just needs to be more broadly available and integrated into build systems.
Would a better example be OpenCL (at least the vision of it)? OpenCL kernels can run on both CPU and GPU, and unlike CUDA, it's not vendor locked. Though I've only really used CUDA, I don't know how comparable OpenCL is these days.
One problem is that although OpenCL can run on both CPU and GPU, OpenCL code that performs well on CPU and OpenCL code that performs well on GPU can be different.