Benoit Daloze

Howdy, I'm on the Shopify team that is working on both pitchfork and a few different performance-improvement projects for Ruby. There's a ton of activity around Ruby performance right now!

I think we're entering a period of increased experimentation and rapid evolution as demonstrated by projects like YJIT[1][2], improved inline caching[3][4] and Object Shapes[5] (also used by V8), and variable-width allocation[6][7], and smaller improvements like better constant invalidation[8]. Significant investments in TruffleRuby[9] are still going on by Oracle, Shopify, and other companies.

And recently, Takashi Kokubun gave a talk at Ruby Kaigi about the future of JIT compilers in Ruby that gives a peek at a whole new set of optimizations Ruby can work on (as well as some performance comparisons against other interpreted languages)[10]. You may be surprised to see how well Ruby (with the JIT enabled) performs compared to Python 3.

All of which is to say, I think there's quite a bit of performance improvement being made in recent Rubies, and that trend will likely continue for quite some time.

update And I forgot to mention that some very notable computer science researchers and their teams are working in the Ruby community now![11]

[1]: https://news.ycombinator.com/item?id=28938446)
[2]: https://speed.yjit.org/
[3]: https://bugs.ruby-lang.org/issues/18943
[4]: https://bugs.ruby-lang.org/issues/18875
[5]: https://bugs.ruby-lang.org/issues/18776
[6]:…

The author pans meta-tracing and shows a graph of TruffleRuby having really crazy runtime behavior. However, it looks extremely suspicious - like the kind of thing you'd see if there was a memory leak and the garbage collector was running constantly.

Most people seem really excited about the results coming out of Graal and TruffleRuby seems to have some really impressive results in general, so the graph is surprising. It's also missing a bunch of details, so it's hard to speak to its voracity - for example, what are the different versions of the runtimes, what flags were used, on what hardware?

As a counter example, there's a different (admittedly old) result where TruffleRuby beats cruby on the same benchmark by 4.38x. https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html

Startup is likely to be worse, because it typically runs a lot of different code for not long and the JIT might not have enough time to optimize much of that. OTOH the JIT needs to learn what the program is doing, i.e., profile it in a sense, and that has a cost on interpreter speed.
Still these numbers are worse than expected, especially [object validation], so if there is a way to reproduce it'd be great if you can open an issue about it.

Persisting the JITed code is what we think can solve the slower startup entirely:
https://www.graalvm.org/graalvm-as-a-platform/language-implementation-framework/AOTOverview/
Also other things mentioned in https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html#warmup

Here, I don't know all the details, just remember having seen some talks about this in the past.

It builds on top of this effort, https://medium.com/graalvm/graalvm-llvm-toolchain-f606f995bf

"From Interpreting C Extensions to Compiling Them"

https://eregon.me/blog/assets/research/from-interpreting-c-extensions-to-compiling-them.pdf