Wilfred Hughes

Happy to read there is some interest in “Markdown 2.0” or “lightweight markup, next generation”. With adoption by big outlets like Github and StackOverflow, and dozens of static site generators built on plain text source files, marked-up in Markdown syntax, Markdown is becoming a _de facto_ general-purpose document _editing_ and _storage_ standard. With the advent of fast client-side parsers, soon it might even substitute html as a document _exchange format_, too (at least for some use cases).

But there is no official spec, development of the syntax grew stale (from the start, thanks to Gruber)¹, and hence the language heavily suffers from fragmentation²: people want to be able to do more with light-markup, so they build extensions, or fork the language altogether. The troubles with Markdown are discussed in various³ places⁴, John MacFarlane built a formalized comparison tool⁵, but as of yet, there is no joint effort to cooperate on a lightweight markup standard.

There are a few interesting projects that dare to forsake Markdown and try their luck: Fountain.io⁶ (a domain specific light markup language for playwriting), SkrivML⁷, Strictdown⁸, and z.m.l.⁹

Then there are, of course, the many predecessors of Markdown, which prove that Markdown (or light-markup in general) is not the imprescriptible privilege of its self-proclaimed benevolent dictator for life, and some of which are still widely used in various communities: Setext (1992), AFT (1999), Grutatxt (2000), AsciiDoc…

> How well do existing VCSs integrate with it?

Git supports external diffing tools really well with GIT_EXTERNAL_DIFF, which you can use with difftastic[1]. Other VCSs are less flexible. For example, I haven't found a nice way of getting a pager when using difftastic with mercurial.

> Did you feel restricted at any point by writing a diffing tool, instead of basing a new VCS around this concept?

Oh, that's an interesting question! Difftastic has been a really big project[2] despite its limited scope and I'm less interested in VCS implementation.

I think text works well as the backing store for a VCS. There are a few systems that have structured backends (e.g. monticello for smalltalk), but they're more constrained. You can only store structured content (e.g. monticello requires smalltalk code) and it must be well-formed (your VCS must understand any future syntax you use).

Unison[3] is a really interesting project in this space, it stores code by hash in a sqlite backend. This makes some code changes trivial, such as renames.

From the perspective of a text diff, an AST diff is lossy. If you add an extra blank line between two unchanged functions, difftastic ignores it. That's great for understanding changes, but not for storage.

[1]: https://difftastic.wilfred.me.uk/git.html

[2]: https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fantastic-diff/

[3] https://www.unison-lang.org/

There was a good point made, that has stuck with me over the years. that our syntax highlighters are highlighting the wrong thing.

They should not be coloring the grammar, we are good at picking out grammar, they should be highlighting the symbols. each different variable and function name should be getting it's own color. that is, the goal is to make it quicker to distinguish different symbols, not that they are a symbol.

But this is much harder than stylizing the grammar so all our tooling sticks with the easy thing rather that the useful thing. Now, I am being a bit mean on grammar styling. It does help quite a bit but I would like to see a symbol matching engine in action to see if that really works.

Unfortunately I don't remember where I read the original post and am unable to attribute it correctly.

update: while trying to look it up I found this https://www.wilfred.me.uk/blog/2014/09/27/the-definitive-guide-to-syntax-highlighting/

I assumed the readers would follow the link to see the original results. GMP is undoubtedly the state-of-the-art of arbitrary-precision arithmetic. The point of this table was to compare bignum implementations provided by different languages, so programs that use GMP bindings are excluded. Probably it would make sense to include the best result that uses GMP, but I think GHC is quite representative of GMP's performance.

I'm not the first one who draw such a comparison. This article goes into some more detail: http://www.wilfred.me.uk/blog/2014/10/20/the-fastest-bigint-in-the-west/

Fascinating, but not surprising that the study concludes that the "effect becomes weaker with an increase in programming
experience".

Maybe of interest is Hughes' discussion [1] of the different forms of syntax colouring. It was discussed on HN before [2].

[1] http://www.wilfred.me.uk/blog/2014/09/27/the-definitive-guide-to-syntax-highlighting

[2] https://news.ycombinator.com/item?id=8378799

> This looks like a lot of work for what amounts to very little end-user benefit.

Oh, but how wrong you are.

At London Emacs meetup Wilfred gave a talk about his motivations for this work. One of the big and most obvious benefits is this:

while porting some feature to Rust, he discovered a bug in Emacs, fixed it and got given commit access for his work.

You can read the transcript of his presentation here:

http://www.wilfred.me.uk/blog/2017/04/30/remacs-talk-transcript/