Mechanical Sympathy

I don't know why the author called this "Multi-process architectures suck :(" when he really meant "I suck at multi-process architectures :(". Look at what he lists:

- "Context Switching"

You can find 4 cores in a trivial computer these days (8 cores with hyperthreading). This means you can have 8 processes without context switching at all, and also suggests that if you don't use multiple processes, you can only reach about 10-20% capacity on a multi-core machine.

- "Per-process overhead"

It's true it has overhead, but you don't have to create thousands of processes, we have light concurrency patterns to use within the actual thread/process.

But even then, you don't even have to run more than one primary process per core. You have multiple cores. They can't together run one process.

- "We built for a single-core world" and "We lack the tools"

Those are outright PEBKAC errors, and not architectural problems. We're way past the stage when multi-process architectures were a hardly understood, confusing problem. We have the right tools, for those who look for them.

Cache misses and overly eager context switching are bad, and you can read a lot about this by Martin Thompson on his Mechanical Sympathy blog: http://mechanical-sympathy.blogspot.com/ There are ways to design multi-process systems to take best advantage of all cores, and your machine caches.

But even Thompson's Disruptor architecture is based around message passing and multiple processes. Because, again, in a…

Here are my guesses (in addition to your points):

- Until late in it's lifecycle, .net runtime performance was worse than JVM.

- Also, the Windows TCP stack, re-architected with Vista, had severe latency issues, which probably led to wholesale abandonment of that platform for low-latency applications. Some of these problems were fixed in Windows 8, but it was probably too late by then.

(These two points are based on my own experience with Windows/CLR/JVM in the mid/late 2000s and some anecdotes I heard second-hand. So I can't really cite any public sources for them)

I'm sure the fact that you can customize Linux to the extreme and the lack of .Net on Linux in those days probably had a lot to do with the choice of Java, in addition to the above.

However, based on my reading of blogs like Mechanical Sympathy [1] a while ago (~8-10 years), there does seem to have been some demand for .Net-based solutions in the HFT space. They had code for both Java and C# in their public code.

[1]https://mechanical-sympathy.blogspot.com/

It's not either/or. Everything needs to be tuned to achieve the final latency requirements. This includes tuning java code to eliminate creating "garbage", immutable string objects mostly, unnecessary object creation, creating short lived objects compared to long lived etc. The other part right sizing jvm generational partitions, using right gc algo etc. Next part would be to find out the code path with lowest latency requirement and making it hot/jitting. Even next part would be to tune OS for things like thread pinning etc etc. You can go further and tune the network layer. Every little thing counts.

You can go even further and employ what's called mechanical sympathy (https://mechanical-sympathy.blogspot.com/) and write your code in a way that makes it easier for the processor to process faster.

I've never worked with application that use off-heap allocations or no-gc requirements.

I didn't feel bruised, I felt pity ;)

People write hard constrained trading software in Java e.g. http://mechanical-sympathy.blogspot.se/ Googling for java avonics has some hits too e.g. http://www.slideshare.net/keugbang/t-50-avionics-embedded-software-development-using-java

I was pretty familiar with the intracities of real-time called iirc "guaranteed service" back when I worked at Symbian. That was a kernel written in dialect of C++. And I've written camera and video capture software for, among others, Android phones. So fairly general experience, I'd say. Still so much more to learn.

You, on the other hand Chris, keep calling out Rob, me and anyone else who thinks that PDP-11 is not a necessary requirement for the next decade's "system language" ;) What systems projects have you worked on then?

Very much unfashionable for the HN crowd. Most of the new research in concurrency is happening in Java and funded by the high performance trading industry - an industry which is very far from the HN crowd. The new Java8 stampedlock is a good example. It's possible to implement it in C++ as well, but because of the guarantees required by the lock it is a very difficult lock to integrate into C++ code. On the other hand, the JRE guarantees the correct constraints for Java code making a stampedlock very easy to use [1]. The performance of a stampedlock also seems to be the best case for any multi-reader environment. [2]

[1] http://concurrencyfreaks.blogspot.com/2013/11/stampedlocktryoptimisticread-and.html

[2] http://mechanical-sympathy.blogspot.ca/2013/08/lock-based-vs-lock-free-concurrent.html

Java 7 and above provide XX:MaxGCPauseMillis flag for GC conf

From Mechanical Sympathy Blog [1]

" G1 is target driven on latency –XX:MaxGCPauseMillis=<n>, default value = 200ms. The target will influence the amount of work done on each cycle on a best-efforts only basis. Setting targets in tens of milliseconds is mostly futile, and as of this writing targeting tens of milliseconds has not been a focus of G1. "

So in Java world we are talking 100s of milli seconds of worst case which is 3 order of magnitude higher than Go.

1. http://mechanical-sympathy.blogspot.com/2013/07/java-garbage-collection-distilled.html