Johnny's Software Lab

This whole article seems to be picking a lot of nits. I didn't read it too deeply, so feel free to correct me if I'm wrong, but the biggest complaints highlighted in the article are:

1. Modern CPUs use instruction level parallelism (ILP)

2. Memory isn't linear (you have separate caches, L1, L2, L3 and main memory)

If you've ever debugged a C or C++ program in release, you've quickly found out about ILP. The code still maps relatively close to the hardware, it just won't run in the sequential order you've provided it in. Many C and C++ programmers know this and try to make the implicit assumptions in their code explicit to allow the compiler to reorder instructions more easily/reduce memory dependency chains[0].

And I'm sure you've heard several proponents over the past few years (Mike Acton comes to mind) espousing data oriented design. This is an entire code methodology intended to help the CPU caches, and it shows an implicit understanding that memory is not linear. Heck, most C/C++ programmers realize that they're using VRAM all the time, and the memory locations they get aren't necessarily backed by physical memory until the OS sorts it out. This is especially transparent if you've ever done any sort of memmapping with files or played with virtual memory.

Anyways, C doesn't necessarily map directly to the hardware, but it's a heck of a lot easier to intuit what a C program will end up doing on the CPU vs what a Python program will do. And most C/C++ programmers…

Hi!

I am Ivica, I am the guy working for Johnny's Software Lab (johnnysswlab.com) - web site and one man company that specializes in software performance. A few times I saw posts from my web site here, that means at least some people find this topic interesting.

I know people don't like sales pitches, but this is exactly what it is, admins please forgive me.

For all software developers who want to speed up their software, I created two vectorization workshops, one which deals with AVX on Intel and AMD CPUs and the other for NEON on ARM CPUs. They are two days long, and cover programming using compiler intrinsics. No knowledge of vectorization is required, but you do need to grasp basic concepts of C and C++ (loops, functions, arrays, bit manipulation).

We had a pilot AVX workshop two weeks ago, and the feedback was very good: the workshop is interesting, challenging but not too difficult and teaches useful things you can immediately use to speed up your software.

The workshop consists of lectures and exercises and we go from essentially no knowledge to everything you need to know about vectorization in two days.

If you are interested in learning about vector programming, you can learn more info about it here, including topics that we will cover, available dates and prices.

https://johnnysswlab.com/avx-neon-vectorization-workshop/

Thank you for your attention and have a good day!

Ivica

Caches and Memory Systems have to be looked at from three related perspectives viz. a) Hardware/SoC/Physical level b) OS/Compiler system software level c) Applications usage level.

To that end;

1) Buy/Beg/Borrow/Steal the Curt Schimmel book mentioned above. That one book will teach you all about how OSes manage caches/memory for SMP multiprocessing systems. You really get to understand multiprocessing at a fundamental level.

2) Memory Systems: Cache, DRAM, Disk by Bruce Jacob et al. - Old classic and a comprehensive book on all aspects of memory systems. Supplement with papers on newer technologies as needed.

3) Ulrich Drepper's classic What every Programmer should know about Memory - https://news.ycombinator.com/item?id=36420502

4) Cache-Oblivious Algorithm - https://en.wikipedia.org/wiki/Cache-oblivious_algorithm See in particular, Harald Prokop's thesis on the above linked to under "References" section.

5) Cache-aware Programming pattern from the book Game Programming Patterns - https://gameprogrammingpatterns.com/data-locality.html

6) There are lots more resources on cache-aware programming and using compiler builtins to help you do it. Here is a list of tips - https://johnnysswlab.com/make-your-programs-run-faster-by-better-using-the-data-cache/ This website has a lot of info. on all aspects of memory optimization.

7) AoS and SoA - https://en.wikipedia.org/wiki/AoS_and_SoA