Matt Pharr

> You really should see if you can get the compiler to auto-vectorize first (possibly padding data structures and loops) before you write anything by hand.

Counterpoint: https://pharr.org/matt/blog/2018/04/18/ispc-origins

> I think that the fatal flaw with the approach the compiler team was trying to make work was best diagnosed by T. Foley, who’s full of great insights about this stuff: auto-vectorization is not a programming model.

> The problem with an auto-vectorizer is that as long as vectorization can fail (and it will), then if you’re a programmer who actually cares about what code the compiler generates for your program, you must come to deeply understand the auto-vectorizer. Then, when it fails to vectorize code you want to be vectorized, you can either poke it in the right ways or change your program in the right ways so that it works for you again. This is a horrible way to program; it’s all alchemy and guesswork and you need to become deeply specialized about the nuances of a single compiler’s implementation—something you wouldn’t otherwise need to care about one bit.

> And God help you when they release a new version of the compiler with changes to the auto-vectorizer’s implementation.

> With a proper programming model, then the programmer learns the model (which is hopefully fairly clean), one or more compilers implement it, the generated code is predictable (no performance cliffs), and everyone’s happy.

OpenMP is a bad approach for the types of problems commonly encountered in games and graphics programming in my experience. Matt Pharr's excellent series of articles on the history of ISPC gives some good explanations of what programming models actually work well for graphics particularly: http://pharr.org/matt/blog/2018/04/30/ispc-all.html

At the time I was doing most of my SPE work (helping to optimize launch titles at EA prior to the launch of the PS3) most titles weren't taking much advantage of them at all. We were a central team helping move some code that seemed like it would most benefit over, I was particularly involved in moving animation code to the SPEs. There weren't really any options for libraries to help at that point, other than things we were building internally, so it was almost all manual work.

Later on in the PS3 lifecycle people moved more and more code to the SPEs. To my knowledge most of that work was largely manual still. For a while I was project lead on EA's internal job/task management library which had had a big focus on supporting use of the SPEs but my involvement in it was mostly during the early part of the Xbox One / PS4 generation. The Frostbite graphics team in particular did a lot of interesting work shifting GPU work over to the SPEs (I think some of it they've talked about publicly) but I wasn't directly involved in that.

> I wonder what the overhead of rounding every operation is? Comparable to the cost of using a proper Decimal class?

For what it’s worth, for the major ICs (Intel, NVIDIA, etc.) there is zero extra overhead. A choice of rounding modes is part of the floating point operation’s instruction. And keep in mind that a floating point op is always rounding no matter what you do, the question is whether it’s always using the same rounding strategy consistently, whether you can control it, and whether it has what you need.

> I remain skeptical

You are correct.

There are good reasons not to use float for money that the article didn’t discuss, and perhaps the author isn’t aware of. You run out of integer precision at 2^24, which is only 16 million. If you process a 20 million dollar payment in units of dollars, you might be off by at least a dollar. That error will multiply with every floating point op done on the result. If your units are pennies, the largest safe value is only 160,000 dollars. If you ever subtract floats, like say make a payment or withdrawal, you can run into catastrophic cancellation without knowing it. Deposit $200k and then withdraw $199k, suddenly you have a small balance with large error that could remain in your account and continue to grow until the balance is zero. https://pharr.org/matt/blog/2019/11/03/difference-of-floats.html

I think one of the best ways to understand the "nice property" of compilers we like isn't necessarily determinacy, but "programming models".

There's this really good blog post about how autovectorization is not a programming model https://pharr.org/matt/blog/2018/04/18/ispc-origins

The point is that you want to reliably express semantics in the top level language, tool, API etc. because that's the only way you can build a stable mental model on top of that. Needing to worry about if something actually did something under the hood is awful.

Now of course, that depends on the level of granularity YOU want. When writing plain code, even if it's expressively rich in the logic and semantics (e.g. c++ template metaprogramming), sometimes I don't necessarily care about the specific linker and assembly details (but sometimes I do!)

The issue I think is that building a reliable mental model of an LLM is hard. Note that "reliable" is the key word - consistent. Be it consistently good or bad. The frustrating thing is that it can sometimes deliver great value and sometimes brick horribly and we don't have a good idea for the mental model yet.

To constrain said possibility space, we tether to absolute memes (LLMs are fully stupid or LLMs are a superset of humans).

Idk where I'm going with this

> ISPC: https://ispc.github.io/ an open source compiler for a SPMD language which compiles it to efficient SIMD code

I've been learning ispc lately and it does seem like a wonderful solution, you avoid having to build separate implementations for every instruction set and/or worrying about per-compiler massaging to get it to recognise the vectorisation opportunities. The arguments for having a domain-specific language variant and why it was written (https://pharr.org/matt/blog/2018/04/30/ispc-all.html is a good read) seem like persuasive arguments.

However, outside of the projects in the above list - it doesn't seem to have very wide usage. There are still commits coming in/responding to some issues so it doesn't seem dead, but there are many issues untouched or just untriaged. There isn't much discussion about using it, or people asking for advice. The mailing list has about a message a month.

Is it merely just an extremely highly specialised domain? Is it just that CUDA/OpenCL is a more efficient solution for most cases where one would otherwise consider it? Are there too many ASM/intrinsic experts out there to bother learning?

Haven't seen it mentioned in here: Matt Phar is also behind ispc, a subset of C (?) in the spirit of SPMD, and an optimizing compiler (based on llvm iirc). It can target Intel AVX (up to avx512, I think?) and is quite fun to work with. It can help out if you're in a rut of using intrinsics and cursed cuda kernels. A fresh take on a programming model.

I recommend taking a look at the amazing story behind it https://pharr.org/matt/blog/2018/04/30/ispc-all .

It's very interesting to see not only the pure tech part, but also the disheartening lack of interest at Intel on the subject. Missed opportunity I think... You can feel a lot of fun turning into a lot of... Sadness? in the end.

Really a big fan of all things Matt Phar, I hope he has the time of his life at nvidia and that they put him to good use!