Linaro Blog

pocl (Portable Computing Language) [1] provides a remote backend [2] that allows for serialization and forwarding of OpenCL commands over a network.

Another solution is qCUDA [3] which is more specialized towards CUDA.

In addition to these solutions, various virtualization solutions today provide some sort of serialization mechanism for GPU commands, so they can be transferred to another host (or process). [4]

One example is the QEMU-based Android Emulator. It is using special translator libraries and a "QEMU Pipe" to efficiently communicate GPU commands from the virtualized Android OS to the host OS [5].

The new Cuttlefish Android emulator [6] uses Gallium3D for transport and the virglrenderer library [7].

I'd expect that the current virtio-gpu implementation in QEMU [8] might make this job even easier, because it includes the Android's gfxstream [9] (formerly called "Vulkan Cereal") that should already support communication over network sockets out of the box.

[1] https://github.com/pocl/pocl

[2] https://portablecl.org/docs/html/remote.html

[3] https://github.com/coldfunction/qCUDA

[4] https://www.linaro.org/blog/a-closer-look-at-virtio-and-gpu-virtualisation/

[5] https://android.googlesource.com/platform/external/qemu/+/emu-master-dev/android/android-emugl/DESIGN

[6] https://source.android.com/docs/devices/cuttlefish/gpu

[7] https://cs.android.com/android/platform/superproject/main/+/main:device/generic/opengl-transport/host/libs/virglrenderer/README.md

[8]…

For reference, [0] appears to be the referenced paper. The relevant passage:

> To understand how disabling these optimizations may impact performance, we ran SPECint 2006 with GCC and Clang, respectively, and measured the slowdown when compiling the programs with all the three -fno-* [-fno-strict-overflow, -fno-delete-null-pointer-checks, and -fno-strict-aliasing] options shown in Figure 9. The experiments were conducted on a 64-bit Ubuntu Linux machine with an Intel Core i7-980 3.3 GHz CPU and 24 GB of memory. We noticed slowdown for 2 out of the 12 programs, as detailed next.

> 456.hmmer slows down 7.2% with GCC and 9.0% with Clang. The first reason is that the code uses an int array index, which is 32 bits on x86-64, as shown below.

int k;
int *ic, *is;
...
for (k = 1; k <= M; k++) {
...
ic[k] += is[k];
...
}

> As allowed by the C standard, the compiler assumes that the signed addition k++ cannot overflow, and rewrites the loop us- ing a 64-bit loop variable. Without the optimization, however, the compiler has to keep k as 32 bits and generate extra in- structions to sign-extend the index k to 64 bits for array access. This is also observed by LLVM developers [14].

> Surprisingly, by running OProfile we found that the most time-consuming instruction was not the sign extension but loading the array base address is[] from the stack in each iteration. We suspect that the reason is that the generated code consumes one more…

> AMD could release a 2x powerful GPGPU tomorrow for half the price and most current NVIDIA users wouldn't care because what good is that if you can't program it?.

Correction: Nobody will be able to use the AMD hardware (outside of computer graphics) because everybody has been locked-in with CUDA on Nvidia. They can not even change even if they want to: it is pure madness to reprogram an entire GPGPU software stack every 2 years just to change your hardware provider.

And I think it will remain like that until NVidia get sued for anti-trust.

> ARM and Intel make great software [..] doesn't open-source any of that either for the same reasons as NVIDIA.

That's propaganda and it's wrong.

Intel and ARM contribute a lot to OSS. Most of the software they release nowadays is Open Source. This includes compiler support, drivers, libraries and entire dev environment: mkl-dnn, TBB, BLIS, ISPC, "One", mbedTLS.... ARM has even an entire foundation only to contribute to OSS (https://www.linaro.org/) .

Near to that, NVidia does close to nothing.

There is no justification to NVidia's attitude related to OSS. It reminds me the one of Microsoft at its darkest days.

The only excuse I can see to this attitude is greed.

I hope at least they do not contaminate Mellanox with their toxic policies. Mellanox was an example of successful Open Source contributor/company (up to now) with OFabric (https://www.openfabrics.org/).
It would be dramatic if this disappear.

Two groups probably have more experience in that than others:

Linaro: https://www.linaro.org/

OpenWRT: https://www.openwrt.org/

Going on a bit of a tangent:

Most of the time these companies just don't care.

If you don't mind getting your hands dirty, then have a look at some vendor kernel trees. For example, a $300AUD router released last year is running 2.6.34brcm - http://www.tp-link.com.au/download/Archer-D9.html#GPL-Code

The 802.11AC card in that router will likely never have open source drivers. I'm pretty sure the routing engine is closed source too, although with 2x Cortex A9 chips I would have thought you could get away with doing it in software.

One thing I was curious about but couldn’t work out from reading online is whether hardware-assisted tracing is supported. (see https://www.linaro.org/blog/coresight-perf-and-the-opencsd-library/ or https://www.youtube.com/watch?v=Gm2ZXIB18PQ for descriptions. This uses the ARM CoreSight/Embedded Trace Macrocells feature. The Intel equivalent is something called Intel Processor Trace. MacOS doesn’t seem to expose this tracing functionality in the way that perf on linux may, and the publically released Darwin source code seems to barely mention it). Has anyone here installed Linux on an M1/2 computer and checked if the hwtracing support works?

http://www.linaro.org/linaro-blog/2011/08/15/enabling-device-tree-support-on-arm-platforms/

The Linaro kernels ("Linaro Stable Kernel" is the keyphrase) boot from devicetrees (aka FDTs). Not sure how much of that is upstreamed, but devicetrees are pretty standard in the ARM world as far as I can tell. (Also, in general if you're doing Linux stuff on ARM, Linaro is the place to look for everything).

Edit: actually, that link isn't very relevant - I just lazily googled "linaro devicetree". But you should be able to find more info!