Shopify engineers write about Ruby and Rails performance, including YJIT, parser work, and Ruby tooling.
Commenters cite it for YJIT's speed gains and the post on Ruby outperforming C, with one clarifying that it compared pure Ruby to a C extension.
Your phrasing could be misconstrued by those not familiar with the history. Shopify hasn't been developing TruffleRuby for 10 years. The entire TruffleRuby project is 10 years old, including the first 12 - 18 months where it was a humble intern project. TruffleRuby (previously called JRuby+Truffle) was initially a research project for testing out optimizations, not a production-quality implementation suitable for deployment. I think it's now a viable deployment target, but that wasn't true 10 years ago.
As you might imagine, there's a lot more to production deployments than simple Ruby compatibility. At Shopify, we've recently modified our CI system to support TruffleRuby and are actively running projects against TruffleRuby in CI now [1]. The TruffleRuby 23.0.0 release coming out in the next day or so includes quite a few compatibility issues we've worked out getting a large Rails app booting. That project is on the order of months, not decades.
YARP will make adoption of TruffleRuby easier. Absent a language specification, implementations like JRuby and TruffleRuby have to match what CRuby is shipping and that by necessity means they lag after CRuby releases. Parser changes are amongst the hardest things to port over. YARP eliminates most of the challenges there.
[1] -- https://railsatscale.com/2023-06-12-truffleruby-in-shopify-ci/
Disclaimer: I'm on the TruffleRuby team at Shopify.
Just-in-time compilation of Ruby allowing you to elide a lot of the overhead of dynamic language features + executing optimized machine code instead of running in the VM / bytecode interpreter.
For example, doing some loop unrolling for a piece of code with a known & small-enough fixed-size iteration. As another example, doing away with some dynamic dispatch / method lookup for a call site, or inlining methods - especially handy given Ruby's first class support for dynamic code generation, execution, redefinition (monkey patching).
From https://railsatscale.com/2023-12-04-ruby-3-3-s-yjit-faster-while-using-less-memory/,
> In particular, YJIT is now able to better handle calls with splats as well as optional parameters, it’s able to compile exception handlers, and it can handle megamorphic call sites and instance variable accesses without falling back to the interpreter.
> We’ve also implemented specialized inlined primitives for certain core method calls such as Integer#!=, String#!=, Kernel#block_given?, Kernel#is_a?, Kernel#instance_of?, Module#===, and more. It also inlines trivial Ruby methods that only return a constant value such as #blank? and specialized #present? from Rails. These can now be used without needing to perform expensive method calls in most cases.
I think llvm is not going to be irrlevent rather the amount of new languages that we see mostly depend on llvm IR.
That helps them to reach multiple architectures which is especially even more important since Apple moved away from x86 to ARM.
Writing two backends for two different architectures would be a lot of work and then lots of platform specific optimizations therefore llvm is the present and seems to be the future for the foreseeable future as well.
Besides that, we or at least I studied and was implied that hand written recursive descendant parsers are inferior way of doing things. Real world usage should always start with a grammer handed over to a parser generator (bison etc) that in turn have LR(1) algorithms driven by tables.
Turns out that a of lot of widely used programming languages are moving away from LR(1) and having their parsers as hand written recursively descendent parsers.
Ruby seems to be switching away from generated LR(1) parsers, more details here [0]
EDIT: References.
[0] https://railsatscale.com//2023-06-12-rewriting-the-ruby-parser/
I believe with version 3.3 Ruby is back in a big way! The language focused on developer happiness and derided for its slowness is slow no more.
YJIT is an amazing technology, and together with other innovations like object shapes and various GC optimizations, Ruby is becoming seriously fast! Big Ruby shops such as Shopify [1] have been running 3.3 pre-release with YJIT and reporting double digit percentage performance improvements.
Personally I'm really excited about Ruby and its future. I can't wait to start working with Ruby 3.3 and using it on my client's production sites...
[1] https://railsatscale.com/2023-09-18-ruby-3-3-s-yjit-runs-shopify-s-production-code-15-faster/
Edit: add percentage to performance improvements.
I'm not too invested in ruby but AFAIK they built a new parser from scratch, that was portable, that means that all the other projects like rubocop, solargraph (or how the ruby lsp is called) and all other ruby implementations can use it, so it will be easier from now on to maintain those projects up to date with the official ruby parser.
You can read more about the parser I mentioned here (but it seems that is has/hat a different name before?) https://railsatscale.com/2023-06-12-rewriting-the-ruby-parser/
C itself is fast; it's calls to C from Ruby that are slow. [1]
Crossing the Ruby -> C boundary means that a JIT compiler cannot optimize the code as much; because it cannot alter or inline the C code methods. Counterintuitively this means that rewriting (certain?) built-in methods in Ruby leads to performance gains when using YJIT. [2]
[1]: https://railsatscale.com/2023-08-29-ruby-outperforms-c/ [2]: https://jpcamara.com/2024/12/01/speeding-up-ruby.html