Codeplay Blog

I recently looked into GPGPU programming, and it seems to be a bit of a mess, which is surprising seeing how long it's been around and how much it is used. I think a large part of it is due to Nvidia using their strong position to hamper anything other than Cuda.

OpenCL 2.0 was announced in 2013, Nvidia added OpenCL 2.0 "for evaluation purposes" in 2017. I don't think they have final support yet.

OpenCL 2.1 uses the same intermediate language as Vulkan, SPIR-V. Nvidia does not seem to support this.

When OpenCL 2.2 was announced in 2017 Khronos said they were working on converging OpenCL and Vulkan compute, but OpenCL will remain separate. I think this means that they will both use the same SPIR-V backend to run shader code, but I don't know.

Khronos also has SYCL. The latest version, 1.2.1, was released a few days ago and uses OpenCL 1.2 as the backend. I'm guessing it doesn't use a newer version because of Nvidia's poor support for them, but AMD doesn't seem to support it well either.

Then there's a multitude of other libraries/platforms, like POCL, HCC, Acoran, that I don't know anything about but found mentions of. AMD has HIP which can convert Cuda to run on AMD hardware, so maybe Cuda is the best option for AMD as well?

I find it a bit of an embarrassment for the industry that it's this messy and complicated, and I don't see it getting better in the near future. It seems as if the proprietary Cuda is still the best option, and that's a big failure for everyone…

Training - I have a feeling it's going to be NVidia for a while. AMD just doesn't show interest in joining the party, and NVidia has been optimising CuDNN for a while now, it will take a bit of time to play catch up. I really don't understand what AMD have been doing the past 2 years here.

Inference - NVidia will have a much harder time enforcing a monopoly here, because they are not, and cannot be the dominant player on all the hardware where neural networks will run after training. ARM, Qualcomm and others in mobile space will be pushing it hard, as will vendors running neural nets on FPGA/ASIC designs that are now emerging.

Will be interesting to see what effect architectures using low precision or binary weights (for both training and inference) will have too on the hardware landscape.

SPIR-V - AFAIK Codeplay (https://www.codeplay.com) are working on SPIR-V support for tensorflow, that should in theory help to use TF on various hardware that supports Vulkan/SPIR-V. But I guess each vendor will still need to tune things like convolution kernels for their specific hardware to squeeze the best perf out

https://www.codeplay.com/products/computesuite/computecpp
Enable SYCL for all openCL devices so Intel, AMD, Nvidia, FPGAs, a lot of things and smartphones which is order of magnitude more devices than CUDA.Products targeting only nvidia devices are mostly niche markets which is pathetic.
As for debuggers codexl has been extended to support it.

Except SYCL there's Open{MP/ACC} gpu offloading which become viable and portable.
There's also HIP/rocm which transpile to openCL AND CUDA (best of both worlds?)
And can transpile CUDA to HIP almost totally automatically.
That's how AMD ported tensorflow to openCL.

If only,

https://www.khronos.org/sycl/

https://developer.nvidia.com/cuda-zone

https://devblogs.microsoft.com/directx/

https://developer.apple.com/metal/

https://os.mbed.com/

https://www.arduino.cc/

https://www.unrealengine.com/en-US/

https://www.crytek.com/cryengine

https://www.misra.org.uk/Activities/MISRAC/tabid/171/Default.aspx

https://www.snsystems.com/

https://www.codeplay.com/

https://docs.microsoft.com/en-us/windows/apps/winui/

https://www.qt.io/

This is the kind of tooling and ecosystems that Rust still needs to build, besides a plain compiler + cargo.