John Myles White

I think people often underestimate (or just plain don't know about) the degree to which a multiple-dispatch-based programming language like Julia effectively implies its whole own dispatch-oriented programming paradigm, with both some amazing advantages (composability [1], and an IMO excellent balance of speed and interactivity when combined with JAOT compilation), but also some entirely new pitfalls to watch out for (particularly, type-instability [2,3]). Meanwhile, some habits and code patterns that may be seen as "best practices" in Python, Matlab can be detrimental and lead to excess allocations in Julia [4], so it may almost be easier to switch to Julia (and get good performance from day 1) if you are coming from a language like C where you are used to thinking about allocations, in-place methods, and loops being fast.

Things are definitely stabilizing a bit post-1.0, but it's still a young language, so it'll take a while for documentation to fully catch up; in the meanwhile, the best option in my experience has been to lurk the various chat forums (slack/zulip/etc. [5]) and pick up best-practices from the folks on the cutting edge by osmosis.

[1] https://www.youtube.com/watch?v=kc9HwsxE1OY

[2] https://www.johnmyleswhite.com/notebook/2013/12/06/writing-type-stable-code-in-julia/

[3] https://docs.julialang.org/en/v1.5/manual/performance-tips/#man-code-warntype

[4] https://github.com/brenhinkeller/JuliaAdviceForMatlabProgrammers

[5]…

For readers who are OK with some math, I recommend John Myles White's eye-opening post about means, medians, and modes: http://www.johnmyleswhite.com/notebook/2013/03/22/modes-medians-and-means-an-unifying-perspective/ He describes these summary descriptive stats in terms of what penalty function they minimize: mean minimizes L2, median minimizes L1, mode minimizes L0.

A single-number statistic is _going_ to leave things out, so if you must boil things down into one number, or even a few numbers, you're going to lose something that you had in the raw data. This is why I find claims along the lines of "statistics don't tell the whole story" a little bemusing - of course they don't, the very definition of a statistic is a summarization of data that is easier to work with. The question is what data is kept or lost, or more generally what importance we place on different aspects of the raw data such that it's reflected in our descriptive statistics.

The lessons for non-technical people who want to communicate with descriptive statistics are to recognize that summarization is inherent in the nature of any descriptive statistic, that they are thereby opinionated in some way in terms of what they've preserved and what they've left out, and to recognize whether those opinions are appropriate for your purpose.

I have extensive work in Matlab, C, Fortran, R, Python, Julia, SAS (IML), and am learning Go and Rust. I have played with Java and Kotlin on non-serious projects.

I typically use Python for data-related tasks.

Julia is just okay. It lost its steam due to long beta, slow time to first plot after 1.0 release, and generally not being a strategically adopted language by ML innovation groups. A great example of what is versus what was hoped is John Myles White's work[0], where the headline is "worked on large community Python projects" but is one of the originators of Julia's fundamental data packages.

I've had lots of intelligent people who use code as a tool instead of as a craft -- for data folks, their craft is working with data and not the tooling around it. For folks like this, Python is simply a better option in today's environment.

[0] https://www.johnmyleswhite.com/about/

Seems like as time goes on, I have to deal with more and more bugs in iOS and OS X. I know I'm not the only one.[1] I think I've figured out why: Apple doesn't track bugs. I just know they don't, even though I don't work there. Its like they just add features and release a new version, so they can ignore all bugs reported for previous versions. Crash reports are automatically and constantly submitted (looking at you Safari), but it never seems to get better. Gives me the feeling that the bugs are never fixed, just carried over into every new version.

1. http://www.johnmyleswhite.com/notebook/2012/10/02/finder-bug-in-os-x/

Okay. This post:

http://www.johnmyleswhite.com/notebook/2013/12/22/the-relationship-between-vectorized-and-devectorized-code/

had worried me a couple of years ago. JMW shows that vectorized was much slower also in Julia (though still both faster than R - but that's not difficult).

Glad to see Julia is very fast in both cases, though it's still somewhat perplexing the extent to which vectorized code is necessarily slower. I'm thinking that the future of GPU enabled languages will mean vectorized code will be faster, so I prefer languages with a bias towards vectorisation.