Michael Malis

Common Lisp is a very pragmatic language, complete with goto, if you want it. This is for the purposes of efficiency rather than because you should use it yourself.

Suppose you wanted a really efficient numerical library written in Common Lisp, you could do it. But you wouldn't want to use map/reduce/remove/etc which are all linear time on the sequences they operate on (which can be lists or vectors). So clean looking code turns out to have poor performance because you have multiple O(n) operations, introducing large constants or if they're nested turns them into quadratic or worse operations. For instance, if we wanted to compute dot product we could, in CL, do:

(defun dot-product (v1 v2)
(reduce #'+ (mapcar #'* v1 v2)))

That's O(n) which is the "best" we can do for dot product. But it actually contains two iterations. In C, you'd have:

double dotproduct = 0.0;
for(int i = 0; i < vector_length; i++) {
dotproduct += v1[i]*v2[i];
}

While for this small example the difference in performance is minor, in the case of a library making many calls to these functions that double loop would add up. And for more complex operations you're introducing higher constants (or worse) by using the clean looking Common Lisp code.

Enter something like Series

Series: https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html

See here for a demo of how it tranforms functional code to imperative: https://malisper.me/loops-in-lisp-part-4-series/

So you still (as the user)…

Here's an article that outlines how recording yourself write code can help with improving productivity: https://malisper.me/how-to-improve-your-productivity-as-a-working-programmer/.

> Watching Myself Code

> One incredibly useful exercise I’ve found is to watch myself program. Throughout the week, I have a program running in the background that records my screen. At the end of the week, I’ll watch a few segments from the previous week. Usually I will watch the times that felt like it took a lot longer to complete some task than it should have. While watching them, I’ll pay attention to specifically where the time went and figure out what I could have done better. When I first did this, I was really surprised at where all of my time was going.

> For example, previously when writing code, I would write all my code for a new feature up front and then test all of the code collectively. When testing code this way, I would have to isolate which function the bug was in and then debug that individual function. After watching a recording of myself writing code, I realized I was spending about a quarter of the total time implementing the feature tracking down which functions the bugs were in! This was completely non-obvious to me and I wouldn’t have found it out without recording myself. Now that I’m aware that I spent so much time isolating which function a bugs are in, I now test each function as I write it to make sure they work. This allows me to write code a lot faster as it…

The main thing I like is basically you always have a debugger present. When you run into some issue you don't have to restart your program in a debugger and recreate it. Even running in production you can remotely connect to your program and inspect everything about it. Another thing about this is that when exceptions happen you're dropped in the debugger before the stack is unwound, and you generally have options to recover and continue execution without restarting everything and losing your state, which imo is the only sane choice for a dynamic language where you know you will be getting some type errors at runtime in development at least. Everyone knows the frustration of starting some python script, having it run for 10 minutes only to have it crash on some exception that you have to fix and restart. Also I'm not entirely sure why, I think because CL does more checks at compile time, and there is more type checking, I tend to get fewer runtime errors and they're closer to the source of the problem and easier to debug than in other dynamic languages. Then there are questions like what happens to existing instances of classes when you add fields or otherwise change them, which CL handles (https://malisper.me/debugging-lisp-part-3-redefining-classes/), but other language REPLs don't.

Regarding the syntax of those type declarations, it's true I also thought CL was verbose and ugly at first, but it is also extremely flexible through macros and reader macros. At the end of…

Maybe start with https://lispcookbook.github.io/cl-cookbook/emacs-ide.html The 'magic' though isn't really that emacs dependent; if you like vim (like I do) you can use vim and still get that amazing experience. https://lispcookbook.github.io/cl-cookbook/editor-support.html has some details on alternatives. Anyway, the magic or synergy is really a combination of: CL itself being designed to support interactive development, a little server you embed (typically automatically if starting from an editor) in a running CL program called Swank, and a standard set of commands to talk to that server (and the underlying Lisp) and interface with your editor called SLIME (https://slime.common-lisp.dev/doc/html/).

The basics of the commands do things like send code over for compilation and redefining things, running things, responding to errors, debugging things, jumping to definition locations.. things you'd normally expect an IDE to do, except because of CL itself so much of that is part of the language and not a separate IDE. Some more editor-specific things come into play that some people really love, and while they may be best done in emacs they're not unique to emacs either. (Example: suppose you messed up the order of your arguments to a function, so it's written like (my-func (list something) some-other-arg) -- editors have support to swap those two arguments in one go, rather than having to backspace and retype or whatever. I personally don't find such features that magical,…

> As someone else mentioned, that sounds more like a work environment problem.

Have you ever used any API by, say, a major cloud provider? For example biquery and google docs are both a buggy mess despite being high profile, long established products. Using libraries like pandas is an exercise in figuring out by trial and error how to get it not to corrupt your data and which of the recommended ways of doing things don't involve a 2 orders of magnitude performance defect.

I'm in fact mostly using "repl"-driven development for these types of problems, but of course it's inferior to what I mean by debugger driven, a crippled subset to be precise. The scare quotes are because most languages don't have real read-eval-print-loops either, but something less powerful.

Unfortunately to the best of my knowledge there are basically only two languages which support this: Common Lisp and Smalltalk (maybe factor or something else really fringe also does) and I haven't used either in years. What I mean by debugger-driven is that you can literally write your whole program without ever restarting it, from the debugger. By contrast python can't even properly reload a changed module definition.

Interestingly it's hard to google good explanatory links, but the point is that you can write some code, run into a problem, and fix the problem right there, in the debugger without aborting the current execution or unwinding the stack.

Common Lisp and Smalltalk both have a bunch of unique…

My relatively amateur take is that the REPL and the debugging experience seem powerful. I would like to know how they compete with other languages.

The REPL is the center of everything and it enables you to change functions on a running program. Tracing a function (shows function arguments on every call) is a simple as calling trace(function-name).
If a program crashes, it does not really crash... it enters some debug mode which offers possible resolutions, including change the function that failed.
The REPL can trivially show you the assembly code of individual functions. You can also add declarations for each function with hints for the compiler (and then check the size of the resulting assembly).

I believe this series of articles highlights some of the debugging features.

https://malisper.me/debugging-lisp-part-1-recompilation/

There is also a story called "debugging code from 60 million miles away".

Other things that I did not manage to explore yet are the macros, which allows you to create your own domain specific languages.

My general impression so far is that it is a really powerful language, for lonely hackers. :)