GPUOpen

> Do you have a hardcore introduction with even more detail / perhaps even with examples of implementations?

Hard to beat the actual technical references when you want something hardcore!

The actual CUDA documentation is good:

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html

In particular, "Section 5: Performance Guidelines": https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#performance-guidelines gives a lot of micro-architectural details, including those nasty "bank conflicts" that people keep talking about. Honestly, the original documentation says it best with just a few paragraphs on that particular manner:

> To achieve high bandwidth, shared memory is divided into equally-sized memory modules, called banks, which can be accessed simultaneously. Any memory read or write request made of n addresses that fall in n distinct memory banks can therefore be serviced simultaneously, yielding an overall bandwidth that is n times as high as the bandwidth of a single module.

>

> However, if two addresses of a memory request fall in the same memory bank, there is a bank conflict and the access has to be serialized. The hardware splits a memory request with bank conflicts into as many separate conflict-free requests as necessary, decreasing throughput by a factor equal to the number of separate memory requests. If the number of separate memory requests is n, the initial memory request is said to cause n-way bank conflicts.

See? Its really not…

I recently looked into GPGPU programming, and it seems to be a bit of a mess, which is surprising seeing how long it's been around and how much it is used. I think a large part of it is due to Nvidia using their strong position to hamper anything other than Cuda.

OpenCL 2.0 was announced in 2013, Nvidia added OpenCL 2.0 "for evaluation purposes" in 2017. I don't think they have final support yet.

OpenCL 2.1 uses the same intermediate language as Vulkan, SPIR-V. Nvidia does not seem to support this.

When OpenCL 2.2 was announced in 2017 Khronos said they were working on converging OpenCL and Vulkan compute, but OpenCL will remain separate. I think this means that they will both use the same SPIR-V backend to run shader code, but I don't know.

Khronos also has SYCL. The latest version, 1.2.1, was released a few days ago and uses OpenCL 1.2 as the backend. I'm guessing it doesn't use a newer version because of Nvidia's poor support for them, but AMD doesn't seem to support it well either.

Then there's a multitude of other libraries/platforms, like POCL, HCC, Acoran, that I don't know anything about but found mentions of. AMD has HIP which can convert Cuda to run on AMD hardware, so maybe Cuda is the best option for AMD as well?

I find it a bit of an embarrassment for the industry that it's this messy and complicated, and I don't see it getting better in the near future. It seems as if the proprietary Cuda is still the best option, and that's a big failure for everyone…

This adds more to the evidence that Vulkan / DX12 seems like a failed API design, when too many graphics engineers are reinventing the wheel by building their render-graph API on top of these (propietary triple-A game engines, Unreal Engine, and now Godot...) Instead of real-time graphics APIs providing all these low-level manual synchronization primitives and cumbersome PSOs, maybe they should just provide an official Render Graph API instead? Provide all of the operations and its dependencies in an acyclic graph up-front, and the driver handles synchronization automatically in the most performant manner tailored to the hardware.

I guess there needed some trial-and-error in the gamedev world for about a decade to really nail down a nice to use but also performant graphics API design. Vulkan being originated from the Mantle API from AMD didn't help - since it was a low-level console API mainly accustomed to AMD's GPUs and really didn't seem like it would fit for a more "general-purpose" API spanning a huge range of hardware and can stand the test of time. And with Microsoft's DX12 hastely copying from AMD's initial design it also has all the same issues (The irony is that DX11 is still the best graphics API you can use in gamedev in Windows in terms of ergonomics and even performance - seeing many trying to dauntingly build a DX12 backend and end up performing worse than DX11...)

Nowadays I'm obversing that the industry has known these issues for a while and are…

> What about generating foliage or varied ground textures? What about generating buildings? Or NPCs?

This isn't how DLSS works at all. DLSS can't create new buildings or foliage, or NPCs, so hard to foresee problems of the kind Valve are concerned with. Same for varied ground textures- the entire point of the technology is to sharpen and upscale the original image, or in case of DLSS3 inject new matching frames.

The only "risk" to DLSS is in Nvidia's own training data, but there is no "risk" of another company's existing IP leaking into final frame - again if there was, gamers wouldn't want to use it, as its destroying the original frame! If the resulting frame isn't a near perfect match for the original, DLSS has failed. Thankfully it does nearly perfectly match the original in use and alongside AMD's FSR 2.0 stuff has been one of the best advancements in gaming technology of recent years - effectively significant FPS boost "for free" on same hardware.

> https://www.nvidia.com/en-us/geforce/technologies/dlss/

> https://gpuopen.com/fidelityfx-superresolution-2/

OpenGL still has lots of steam in it and 4.6 was recently released, so it isn't going away anytime soon.

Even on mobile devices, Vulkan is hardly being embraced by Android developers, because it requires Android 7 or later, while being an optional API, which implies a tiny portion of their customer base.

So very few devices support it, to the point Google doesn't think it is worthwhile to display on Android dashboards.

https://developer.android.com/about/dashboards/

http://vulkan.gpuinfo.org/vulkansupport.php#android_devices

AMD is also busy trying to cater CAD vendors to adopt Vulkan, with initiatives like V-EZ.

https://gpuopen.com/v-ez-brings-easy-mode-vulkan/

However, if you want to be an engine developer, probably it would be better to have Vulkan knowledge than the APIs that everyone else already knows.

Some guys at Khronos sessions (check their YouTube) stated they would favor candidates in interviews in such cases.

For product development, I am a bit fan of middleware where each API is just a tick box away, though.

Not sure if I completely understand what "Ryzen AI" does, but tinygrad for example has some limited support for RDNA3[0]. It isn't quite there yet in matters of performance though, as you can read in the comments of that file.

There's also a small tutorial by AMD on how to use the WMMA intrinsics[1] using their hipcc[2] compiler. Documentation is sparse kinda sparse, but the instruction set is not huge. The RDNA3 ISA guide[3] might also be helpful (and only a fraction of the pages are relevant.)

So it doesn't seem like RE is necessary? AMD dumped most code on GitHub.

0. https://github.com/tinygrad/tinygrad/blob/master/extra/gemm/hip_matmul.py

1. https://gpuopen.com/learn/wmma_on_rdna3/

2. https://github.com/ROCm/HIPCC

3. https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna3-shader-instruction-set-architecture-feb-2023_0.pdf