Paul Khuong, a systems programmer, writes about low-level performance, SBCL internals, concurrency, and algorithms like binary search.
Commenters call his work worth skimming, cite his posts on memory-ordering fences and binary search, and use his SBCL hacking guide as a reference.
I do get that, as an amateur Lisper.
I have played with CCL and SBCL on and off for a while. Ironically, the AUR package I used Clozure Lisp with does not even exist anymore on Arch AUR repos.
https://aur.archlinux.org/packages/ccl-bin/?comments=all (That won't work)
I get the trade off, but if you read SBCL dev blogs, like PVK, you will recognize really competent programmers and elegant solutions.
https://pvk.ca/
The problem is these gods (I revere them) cannot possibly encode all the hacks and knowledge. They have to code, and the nuances of such "hacks" are internalized by them and forgotten. Ironically, a lot of shit talk on Lisp transpires here, and it is one of the most sophisticated developer-productive tools I have seen, with many laudable features other toolchains brag about. Lisps had them for years. This is not to be an elitist jackass; the cultural shift to obsolence (sadly, my view) is why only diehards know and others avoid implementation details and functionality: we don't use it, so we don't care.
Either way, dead code is scary, especially in such scenarios with binary image dumping and bit fiddling hacks. I was ironically aware of such issues with SBCL when I stumbled on Gentoo Hardened list info when people explained how Lisps have trouble in such constrained environments.
https://archives.gentoo.org/gentoo-hardened/message/d2fb14f123e7db18dc7fdbd7ba4c74fa
I worry later on such methods leave blobs of weak executable routines and other items of…
Yeah, so with capabilities, the naive idea of "categories" might or might not make sense. It depends how much duplication you get (how many pieces of equipment have large counts of capabilities). Most real-world scenarios I've seen only have 2-10 capabilities per piece of equipment though, which makes the simplest, dumbest solution work in the real world (anything fancier pushes closer to the database literature, and there are good solutions, but you hopefully don't have to care).
> tens of thousands of tasks
Am I misinterpreting, or are you saying that you have tens of thousands of queries and a much lower equipment count? That enables all kinds of solutions. Even on a single machine, multi-thread it. For deletions and insertions, use something like bounded-wait hazard pointers [0] with a bias toward cheap reads. If you have concurrent tasks, horizontal scaling is the name of the game. 10 microseconds is an easy baseline for intra-datacenter networking, easily allowing 10 millisecond reads and mutations even with a distributed solution.
Related, simply caching results might get you the performance you want on a single core if you only have hundreds of pieces of equipment you're searching each time. For each capability, just record the last time you mutated it. You can re-use any query result whose associated results don't have potential mutations. Especially if writes are much less frequent than reads, and especially if you have a bias toward not handing out equipment…
Here are a bunch of blogs from my RSS reader that I think are worth reading. Some of them haven't updated in a while, but the archives may be worth a look anyway.
* Blogs with a math focus:
http://blog.sigfpe.com
http://bit-player.org
http://mathr.co.uk/blog/
https://www.johndcook.com/blog/
* Blogs with a programming languages focus:
https://existentialtype.wordpress.com
https://bluishcoder.co.nz
http://blog.ezyang.com
http://akkartik.name
http://lambda-the-ultimate.org
http://mrale.ph
* Blogs with a UI design focus:
http://www.brandonwalkin.com/blog/
http://ignorethecode.net/blog/
http://interuserface.net
* Blogs with a graphics / 3D rendering focus:
http://c0de517e.blogspot.com
http://diaryofagraphicsprogrammer.blogspot.com
https://directtovideo.wordpress.com
http://graphicrants.blogspot.com
http://www.reedbeta.com
http://www.realtimerendering.com/blog/
https://blog.demofox.org
http://eelpi.gotdns.org/blog.wiki.html
* Other interesting / more general programming blogs (many of these have a "systems" focus):
http://danluu.com
http://ewontfix.com
http://chneukirchen.org/blog/
http://neverworkintheory.org
http://apenwarr.ca/log/
https://www.snellman.net/blog/
https://www.pvk.ca
http://psy-lob-saw.blogspot.com
https://randomascii.wordpress.com
http://fastcompression.blogspot.com
https://fgiesen.wordpress.com
http://blog.plover.com
http://chneukirchen.org/trivium/
http://rachelbythebay.com/w/
https://github.com/dym/movitz is a Common Lisp system that runs on bare metal x86. The source code is quite readable.
The rest of this post is an excerpt from an email I sent 6 years ago.
The following comments on runtime systems are partially based on a long c.l.l thread with posts by Lucid, Symbolics, and Franz alumni.
Franz uses a 3-layer approach: CL, a low-level Lisp, and C.
Lucid started with Lisp that generated assembler but reluctantly added some C.
Symbolics Lisp Machines used bootstrap code in a Pascal-level language with prefix syntax. A Symbolics alum said that in retrospect they should have used C.
Most Lisp implementations have subprimitives - low-level functions that can circumvent the type system, often with a prefix such as % or :.
Assembly language integration dates to Lisp 1.5 and there are several common approaches.
1. turn the optimizer off - this is easy to use and implement.
2. optimize the assembler block - Naughty Dog GOAL did this.
3. annotate the assembler block with pragmas that indicate side effects. This can be error prone and difficult to use.
https://www.pvk.ca/Blog/2014/08/16/how-to-define-new-intrinsics-in-sbcl/ is an example of this approach.
Edited to add the SBCL intrinsic link.
Not every programmer needs to be a hardcore systems hacker; you can ignore the low-level details and lead a healthy productive life. But me, personally, I get a massive inferiority complex if I don't dive into Nasm and GDB every 2-4 months. It would pain me not to know useless things about obscure stuff.
Yesterday I was prancing through reddit when I followed this link: http://pvk.ca/
My stomach sank when I realized I wouldn't have the time to "follow along" (i.e. waste the day reproducing his results, checking facts, researching and following links) because I am about to start a lead-dev/PM position and need to work on SaaS and EA stuff. So I closed the tab and moved on to focus on what matters. It wasn't until 1AM that I realized what I have done; aarongough sent me a link to his toy prototyping language and I have just spent 5 hours reimplementing it in Lisp, in the process re-digging into the Self papers! (and discovering warts, but that's another story.)
I am an Adderall prescription away from leading a healthy productive life, without unnecessary knowledge about stuff.
In embedded systems using MCUs with very little memory (AVR comes to mind) assembly might be preferred over high level languages (very generously counting C as HLL here). It might even be fun (for truly launch-and-forget projects -- don't even think about maintaining such code) to forgo structured programming and get creative with control flow and self-modifying code.
As a systems engineer it's good to know _some_ x86-64 assembly as sooner or later you're facing a stack trace / register dump and will have to try to make sense of it. Personally, I would take it only further, if you're interested in building compilers.
As an odd twist to the whole assembly vs. HLL dichotomy, sbcl is used by some as assembly playground: https://pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard/