Vidar Hokstad writes about compilers, Ruby, and software tooling, notably a long series on writing a compiler in Ruby from the bottom up.
Commenters praise his compiler series as a great set of posts, and one says it made compilers make sense to them where the dragon book did not.
I have this "meta-language" rant brewing for my blog[1], and I feel like you took the words out of my mouth!
Are you saying that ANTLR makes some unwarranted assumptions about Unicode? Does that just depend on the generated Java code, and would it be different for generated parsers in C++? I don't know the JVM very well, but my understanding was that the JVM makes some Unicode assumptions that aren't always appropriate.
The last "rant" was here, about meta-languages only being suitable for toys: https://news.ycombinator.com/item?id=13040682
vidarh also responds with his problems parsing Ruby (a "real" language): http://hokstad.com/compiler
The blog is based on my experience in writing a parser for bash. I ported the POSIX shell grammar to ANTLR, but it fell ridiculously short of being a production quality parser.
I believe it's essentially impossible to write a bash parser with ANTLR. But I don't hear about any alternatives. All I heard is yacc/bison for bottom up parsers, and ANTLR for top-down. Shell needs a top-down parser because it's an interactive language (the PS2 prompt) and for completion. But it doesn't appear there is any other "serious" meta-language for top-down parsers other than ANTLR? It certainly is the best documented and longest-lived, but I am surprised how far short it falls for many tasks.
Part of the rant will be a survey of "real" language parsers (i.e. take the top 20 TIOBE language implementations) and see that they almost all use…
https://jrock.us/
https://jgc.org/
http://kaledavis.com/
https://aegir.sexy/
https://www.gwern.net/
https://jacquesmattheij.com/
https://danwin.com/
https://www.kalzumeus.com/
https://www.magic.do/
https://edweissman.com/
https://jakeseliger.com/
http://www.fogus.me/
https://www.stavros.io/
https://jasonlefkowitz.net/
https://minimaxir.com/
https://www.robinwhittleton.com/
https://ciexinc.com/
https://rongarret.info/
https://williampietri.com/
https://bpier.re/
https://joshtriplett.org/
https://ooer.com/
https://feross.org/resume/
https://www.brandons.me/
https://benjaminpbreen.com/
http://hokstad.com/
https://www.scott-a-s.com/
https://www.bermita.com/
http://tlrobinson.net/
https://ldpreload.com/
https://dotat.at/
http://xavvy.com/
https://correa.in/
https://adriann.github.io/
https://leclan.ch/
https://joeyespo.com/
https://pius.me/
https://adamierymenko.com/
https://drewdevault.com/
https://battlepenguin.com/
https://chris.improbable.org/
https://www.snellman.net/
https://ocdtrekkie.com/
https://www.cryptolotus.com/
https://www.gregkogan.com/
https://david.kitchen/
In no particular order. From: https://jessimekirk.com/blog/hn_users_links/. I went through about half of the list looking for personal websites to use as inspiration for my own.
Also cool to play around with are the various options that let GCC (and presumably LLVM) show you the various compilation stages of your C code. You can even spit out the C and resulting assembly side-by-side.
I haven't had the time to play with this, but Part 4 of the excellent "Unix as IDE" series [1] goes into it and I'm sure there's more around the web.
Another really fun way to get into the underlying assembler that the C compiler generates is Vidar Hokstad's "Writing a compiler in Ruby, bottom up" [2]. This series involves writing little C functions, compiling them to the simplest assembly you can get, then writing a ruby compiler (in Ruby!) that emits that assembly. Some people have objections to the approach, but it's really quite nifty. I especially like it because it's really refreshing to see a compiler tutorial that doesn't start with the lexer and parser.
[1] http://blog.sanctum.geek.nz/unix-as-ide-compiling/
[2] http://www.hokstad.com/compiler
This is not really accurate.
When using java, you will not use JSP, JSTL, JSF, Play, Wicket, STripes, Tapestry, etc, all at the same time. Unless you are crazy.
The same way, in Ruby, you will not use ERB, Haml, Erector, Markaby, or the other (at least) 19 template engines out there (http://www.hokstad.com/mini-reviews-of-19-ruby-template-engines.html)
But this comparison is not actually fair, because you mixxed template engines with frameworks. To be fair, you'd have to add Sinatra, Camping, Rails, Merb, Ramaze, Padrino and Hobo to the Ruby list, and CakePhp, Zend and Symfony (at least) to the Php list.
It is not which language has the most options of framework/template engines, but which has the best optins. JSP is one of the worst template engines on Java land, and this is why it is not used as much (well, there is also the fact that it has a very bad reputation, and JSF is considered its successor, but it is worse nonetheless).
I was speaking metaphorically of Pascal being the LLVM of its day, more that it was a known base, portable, easy to explain and the first tool someone would reach for.
Templated IR code that has been LZ compressed sounds excellent, Thumb++ or uop buffer engine. It sounds like from the post below, that the instructions could even be pathologically the uop or VLIW instruction form and the SDE will effectively discover the compact CISC for your specific workload. Why not push the decode engine into the instruction decoder?
I found this awesome post that covers SDE, http://hokstad.com/semantic-dictionary-encoding
I think one way to explore this would be to implement compressed binary loaders, like done for demos etc, but target wasm, wrap it in a decompressor and have the wasm program decompress itself as it executed perhaps on a method by method basis.
I wrote one, though it kinda got away from me (I started very simple, but then started turning it into a Ruby compiler):
https://hokstad.com/compiler
On that note, while there's no new part, after finally revisiting it for the first time in a couple of years, it now compiles itself (from Ruby into 32bit x86).
A few very critical bugs that I'm working on, and a lot of big omissions (no Regexp yet; the Ruby Regexp engine source is bigger than my entire compiler, nor any Floats; syntax lacks a lot of newer features) so it's very much not a usable Ruby at this point. Maybe one day if I get more time.
If you find it interesting, I'd recommend looking at the earlier parts rather than the current source. Lots of the recent changes were very aggressively working around bugs to move it towards self hosting, and it's not pretty.