Gregory Gundersen is a machine learning researcher who writes explanatory posts on statistics, linear algebra, probability, and the history of language models.
HN commenters praise his explanations of moments, SVD, and matrix inversion, with one calling the SVD post their favorite, and note his use of KaTeX.
This article is a bit rough around the edges, and they really should make it clear when they mean "pseudorandom" vs "random", especially when one goal of the article is ostensibly to explain pseudorandomness to laymen!
As an aside, using a source of random numbers to generate other random objects is quite an interesting problem! For converting from one probability distribution to another, we have classical methods like rejection/importance sampling, and modern deep learning makes constant use of the "reparameterization trick" [1,2]. Looking beyond just numbers, here's a neat trick for generating a random orthogonal matrix:
A = np.random(n,n)
Q,_ = np.qr(A)
With a small correction [3], we can guarantee that our samples have uniform distribution (Haar measure) on the space of all orthogonal matrices:
def haar_measure(n):
# naive method
Z = randn(n,n) + 1j*randn(n,n) / np.sqrt(2);
Q,R = np.linalg.qr(Z);
# correction
d = np.diag(R);
PH = np.diag(d) / np.absolute(d);
return Q @ PH;
For more, read about random matrix theory! As an exercise for the reader, how would you generate: a random symmetric matrix? a random skew-symmetric matrix? a random positive-definite matrix? a random hamiltonian path in a fixed graph?
[1]: http://gregorygundersen.com/blog/2018/04/29/reparameterization/
[2]: https://timvieira.github.io/blog/post/2014/07/31/gumbel-max-trick/
[3]: http://www.ams.org/notices/200705/fea-mezzadri-web.pdf
This seems to be a static site generator.
http://gregorygundersen.com/blog/2020/01/12/why-research-blog/ has a HTTP header last updated on 01/13/2020 04:34:31.
http://gregorygundersen.com/blog/2019/12/23/random-fourier-features/ is 01/13/2020 04:34:30, or 1 second earlier.
Also the source code has a link to http://gregorygundersen.com/css/markdown.css which 404's.
If anyone was frustrated like me on this concept of "moments" I found the following insightful and fascinating:
https://gregorygundersen.com/blog/2020/04/11/moments/
I have no idea how I got through undergrad and graduate school without internalizing this concept that seems so foundational. Whacky
Not an application, but this my favorite explanation of SVD:
http://gregorygundersen.com/blog/2018/12/10/svd/
It's such an important operation that I'd say understanding it changes how you understand a lot of linear algebra. That Kun post is also good.
I (shamelessly) stole the layout from Gregory Gundersen's wonderful research blog (https://gregorygundersen.com/blog/). He has instructions on how to replicate it here https://gregorygundersen.com/blog/2020/06/21/blog-theme/
Re: the first sin, I was told to not invert matrices but never given a satisfying reason. So I wrote a blog post about it:
http://gregorygundersen.com/blog/2020/12/09/matrix-inversion/