Stressgrid's blog publishes load-testing benchmarks of web servers in Erlang, Elixir, Go, Java, and Node.js, plus BEAM performance notes.
Commenters link its benchmarks as a reference on Erlang and Cowboy performance, though one notes they are dated.
> I picked up _Programming Erlang_ because I like Joe. I'm trying to understand why the platform, and the accompanying language(s), aren't more widely used? People seem to speak very highly about Elixir, BEAM, and OTP. It all seems really great...
Watching videos of Joe talk super impressed me. He kept the same enthusiasm for programming as any college age programmer. That's impressive!
> From @hajile
>>BEAM is about an order of magnitude faster than Cpython ...If you prefer Ruby syntax over Python, there is no contest here.
>>BEAM is slower than the JVM, but much more stable.
> The second quote seems to be a pretty reasonable trade off (my next question is, how much slower?).
There was a recent web-server benchmark that illustrates the performance and stability trade-off pretty well [1] (or older web-socket one [2]). The StressGrid benchmark shows Erlang/Cowboy achieving about roughly 1/2-1/3 the speed of the compiled options (Java & Go), and roughly double that of NodeJS. Erlang/Cowboy maintained very stable latency numbers after hitting it's peak however (see graph in [3]). There's an oddity with Cowboy 2, that appears to be due to HTTP/2 support but can be disabled (though HTTP/2 in theory should support faster client responses with less multiplexing of HTTP requests).
In general I'd say BEAM is on average faster than CPython, though recent CPython 3.x versions have improved a lot. Importantly BEAM is one of the few dynamic language/VM's that supports multi-core…
I have commented this in other places but the main explanation for the benchmark you linked is that Phoenix (and its underlying Cowboy webserver) are not optimized for this particular workflow.
For a plain-text benchmark, you are ultimately measuring the ability of sitting on top of a socket and reading/writing to that socket as fast as you can.
However, Phoenix/Cowboy, whenever there is an HTTP connection, spawns a VM light-weight process to own that connection, and then each individual request runs in its own light-weight process too. This comes with its own guarantees in terms of state isolation, reasoning about failures, and you get both I/O and CPU concurrency for free - if you were to do any meaningful work on those requests.
Even though the VM processes are cheap to spawn and lightweight, it is overhead compared to something that is just directly reading and writing to the socket. I actually wrote a proof of concept where we just sit on top of the socket without spawning processes and it performs quite well - although I don't think it has any practical purpose. There is also an interesting article from [StressGrid](https://stressgrid.com/blog/cowboy_performance_part_2/) that shows how removing the per-request process speeds it up by ~60%. But once you are going through long requests (i.e. 1ms because you need to talk to a database or API, encode JSON, etc), these differences tend to matter less and the concurrency model starts to give you an upper hand.
Here's a good comparison between Elixir, Go, and Node:
https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/
Some of the things I like better about Elixir and Erlang vs. Go and Java are:
- In Elixir/Erlang, all processes are stored in private memory and can only be accessed via defined message interfaces. In both Go and Java, processes are stored in public memory.
- Elixir/Erlang processes are a lot smaller, memory-wise, than Go/Java processes.
- We moved all of our development at my current company from Java to Elixir because Elixir, being functional, makes writing multi-threaded applications a lot faster/easier. Elixir is also cheaper to deploy. At least that has been our experience.
- I like the syntax of both Elixir and Erlang better than Go, Java, or JS.
- I don't have as much experience with Node/Express, but in my limited testing with Node/Express vs. Elixir/Phoenix, the latter is several times faster and much more scalable. It also makes better use of hardware (which I believe is discussed in the link above).
I hope at least some of this helps answer your question.
As far as from my tests and what I've seen reported online, Go and Rust have a substantial lead (20% ish) over Erlang for high throughput servers.
EDIT: I believe this is partially due to Go being a lot more CPU efficient overall than Erlang (see below). So for simple servers, Go and Erlang will match performance, but for slightly more complex web servers that need to crunch some data, Go [and Rust] will outperform the Erlang VM. https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/
busy wait spins your cpus.
I don't know for sure, but the benchmarks you linked do some more RCA, which involves looking into why cowboy2 is not great (tl;dr HTTP2 support is hard)
also, here is how stressgrid can POC (possibly too dangerous for prod?) to 100k cps (which is on par with what Go can do in the article you linked), you could recompile kernel and configure ranch differently, would probably need to be proven out more:
https://stressgrid.com/blog/100k_cps_with_elixir/
We can't compare about CPU values
Can you explain more why we can't compare? Looking at this chart...
https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixir/10K-cpu.png
...shows pretty clearly how the CPU utilization grows linearly (ignoring some sawtooth) as the load increases, plateaus once the load remains constant, and then comes down linearly as the load decreases on the other end. Looks like a very clear mapping between work done and CPU load to me.