Jeff Preshing

Disclaimer: I haven't implemented this yet, this is based on skimming the paper and what I know about lock-free structures.

Your run-of-the-mill lock-free code basically operates under the premise of "keep trying until something sticks." For example, with only CompareAndSwap you can implement lock-free bit-field operations as:

do {
oldValue = Read(variable);
newValue = oldValue | 0x01;
} while ((CAS(variable, oldValue, newValue) & 0x01) == 0);

You can see that we have to keep looping until the new value (which includes the flag) "sticks." The race is allowed to occur between the Read() and the CAS(), where CAS() is used to determine if a race occurred. CAS will check the current value of the address and update it only if it matches (as an atomic operation on the CPU). The worst case here is O(∞) - if another thread always wins, this loop will never exit. The run-of-the-mill lock-free stack code looks quite similar to this (you're basically updating the head until it sticks). This means that in certain circumstances, lock-free code can actually be slower. I have run into this myself[1]: that code uses locks because I couldn't figure out a faster lock-free implementation (that code is 30x faster than my best lock-free effort).

There are two operations on a stack: Push() and Pop(). Classically, these both compete to write the head (write-contention). The advantage with this algorithm is that only Push() competes to write the head so you're decreasing the…

You might be being really subtle about python there ! I realize that one can create lines of code like :_ = (
255,
lambda
V ,B,c
:c and Y(VV+B,B, c
-1)if(abs(V)<6)else
( 2+c-4abs(V)-0.4)/i
) ;v, x=1500,1000;C=range(vx
);import struct;P=struct.pack;M,\
j ='<QIIHHHH',open('M.bmp','wb').write
for X in j('BM'+P(M,vx3+26,26,12,v,x,1,24))or C:
i ,Y=_;j(P('BBB',(lambda T:(T80+T9
i-950T 99,T70-880T18+701
T 9 ,Ti(1-T452)))(sum(
[ Y(0,(A%3/3.+X%v+(X/v+
A/3/3.-x/2)/1j)2.5
/x -2.7,i)*2 for \
A in C
[:9]])
/9)
) )
(stolen from : http://preshing.com/20110926/high-resolution-mandelbrot-in-obfuscated-python/)

which could be seen as retrograde vs assembler (but not really for the very funny and brilliant code above - you have to see in formatted nicely and run it to realize that there are some great people out on the web!) perhaps in fact I would agree with this dig - some people do write horrid bits…

Here are some software engineering blogs from my RSS reader in addition to the ones listed in TFA. I honestly don't know how over or underrated they are, but I guess that varies depending on which communities you're closer to.

Mikola Lysenko https://0fps.net

Graphics programming, geometry and algorithms.

---

Steven Wittens http://acko.net

Not underrated in any sense of the word, but I have to include it in case there are people who haven't seen it yet. It's about math and graphics, but also about art and design. Amazingly educational.
http://acko.net/blog/how-to-fold-a-julia-fractal/ is a good starting point for new readers.

---

Mark Nottingham https://www.mnot.net/blog/

Networking and HTTP.

---

Jeff Preshing http://preshing.com

Efficient concurrent data structures.

---

Bruce Dawson https://randomascii.wordpress.com

Software efficiency, performance optimization, fractals and more.

---

Yann Collet http://fastcompression.blogspot.com

Modern compression algorithms.

---

Gil Tene http://stuff-gil-says.blogspot.com

Also definitely not underrated, at least not in the Java world. But any engineer who cares about latency and honest benchmarks will benefit. Low latency techniques, GC performance, lock-free programming.

---

Tony Finch http://fanf.livejournal.com

C and UNIX programming, and more.

---

Andy Wingo https://wingolog.org

Scheme, Compilers, JavaScript, and much more.

---

Aleksey Shipilёv http://shipilev.net

JDK internals, low-level optimization.

> It reinforces the idea of Memory Ordering as voodoo

I thought there were some very solid bits in there, like the reminder that memory barriers always come in pairs.

> it conflates compiler optimizations with memory reordering

Well, the code as written is subject to reordering both by the compiler and the CPU. The author is just being honest that this isn't a perfectly crafted example. You could make a version not subject to reordering by the compiler, either by marking variables as volatile or by inserting "compiler barriers" into the code (e.g., empty volatile asm statements). I don't think this is conflation.

> Worse, it's just not true --- while this particular example might not cause problems,

I think what happened is that you interpreted the author's statement as one about memory reordering in general on the x86 architecture, but I think the author is referring to this specific application, which makes the statement true. (The sentence you quoted is bracketed by two sentences which are more explicitly about this demo application, but it's not explicit in the sentence you quoted.)

> memory reordering is still an issue that needs to be dealt with on x86.

The article links to another post by the same author with a similar experiment demonstrating reordering on x86. The example is a lot harder to follow, though, since x86 only allows reordering of stores and loads relative to each other.

http://preshing.com/20120515/memory-reordering-caught-in-the-act

>…

On the contrary, _not_ expecting at least 20% per year (more for generations farther apart) is actually only a recent take for CPUs [1][2] which formed in what I like to call "the great stagnation" of the mid 2010s where AMD bombed, ARM was still a low performance mobile play, and Intel didn't have any stiff competition. After that things have started to pick up a little again now that there is innovation on said fronts [3]

I remember there was even a period Moore's law was commonly conflated with "a doubling in performance every 2 years" (implies people being used to an increase of at least 41% per year) instead of a doubling in transistor count. Because Intel's tick-tock model started ~5 years prior to the great stagnation people commonly claimed it's the end of Moore's law and that's why we have tick-tock's now. Of course these days it's common knowledge that Moore's law (the actual transistor count version) has been holding steady all these years still and that the performance lull wasn't related to hitting and innate technical scaling barriers.

Marketing departments are definitely going to be marketing departments but they aren't the origin of the idea CPUs can have more than minor increments in performance each year.

[1] https://preshing.com/images/integer-perf.png

[2] https://preshing.com/images/float-point-perf.png

[3] https://i.imgur.com/HlGbJE9.png (note this chart is linear not log)

There must be some kind of communication error going on. I don't know much about RCU, so I just pulled up this webpage:

https://www.kernel.org/doc/Documentation/RCU/whatisRCU.txt

In it is:

> The rcu_read_lock() and rcu_read_unlock() primitive read-acquire and release a global reader-writer lock.

Seems like RCU-operations in the Linux kernel are defined in acquire-barrier and release-barrier terms. I heard a while ago that RCU could be discussed in terms of release-consume semantics (which are slightly faster but harder to understand...) but very few people understand release-consume.

As such, release-acquire is probably the memory model of the future. I'm not really aware of anything aside from: Fully Relaxed (unordered), the obscure release-consume, release-acquire, and finally sequentially consistent (too slow for modern systems)

---------

Are you perhaps confusing "acquire-release" semantics (which is a memory-barrier / cache coherence principle) with spinlocks perchance? Acquire-release seems to be the "Fastest-practical" memory consistency model. (Since Relaxed doesn't work, and release-consume is too confusing)

For more info on acquire-release, Preshing's blogposts are great: https://preshing.com/20130922/acquire-and-release-fences/