Kevin Bryan

No disrespect to the parent, but I would strongly discourage any aspiring computer scientist or software engineer from concluding that the above list of topics is in any way remedial or “lower division”.

Google exists in no small part because just over 20 years ago people who could also code and raise money connected just a few of the dots on the deep themes connecting those topics: https://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf

Just over a decade ago Netflix paid out a million bucks (that was real money then) for a modest improvement over their existing recommender based on an SVD-like gradient descent for a low-rank approximate factorization: https://en.m.wikipedia.org/wiki/Netflix_Prize. Many of the competing teams had exemplary academic and industrial pedigree.

Many (if not most) ML practitioners would profit from thinking more about the information-theoretic links between lossy compression, regularization via norm, and channel capacity to this day.

If I correctly gather that this is a syllabus for elite undergraduates, they’ll be grading on a curve.

Here's a pretty good lecture on exactly that subject:
https://www.rose-hulman.edu/~bryan/invprobs/jpegtalk2.pdf

Honestly though, the Fourier transform is useful anywhere that it's easy to think in terms of frequency instead of time. 90% of use cases in the real world come about from trying to do something that's that's easy to express in terms of signal frequency and difficult to express in terms of signal time/space. You can just use the Fourier transform to convert between them efficiently.

To give a morbid, but non-obvious example, I once did some work with a charity that was concerned with migrant deaths. They wanted to figure out where the migrants most at risk were working to approach the farmers hiring them directly. We got a dataset of when migrant bodies were found and did an FFT (among other processing) to find the periodicity of the crops that they were coming to harvest. There's only a few major crops and they tend to have distinct growing periods, so this is enough information to pinpoint certain crops like strawberries.

Google search is the ultimate example (IMO) of an effective, simple, and powerful Markov chain algorithm.

For anyone interested in this stuff, check out "The $25 Billion Dollar Eigenvector"[0]. I did a simple page rank experiment in an IPython notebook[1], demonstrating how PageRank works based on some UBC lectures.

Markov Chain Monte Carlo sampling is the basis of many Bayesian inference techniques, and Markov chains also show up extensively in classic speech recognition pipelines, under various forms of Hidden Markov Models (VB-HMM, GMM-HMM). This stuff forms the foundation of statistical machine learning!

Great post - this simple example serves as a great introduction! Of course, the rabbit hole is deep...

[0] http://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf

[1] http://kastnerkyle.github.io/blog/2014/04/16/simple-page-rank/

This version is actually the full version. The short version was designed for printing out/conference while longer version is designed for hosting on web.

Also there is a book on the subject
Google's PageRank and Beyond
http://books.google.com/books?id=KsHTl_2Pfl8C

Other interesting articles:
THE $25,000,000,000∗ EIGENVECTOR
http://www.rose-hulman.edu/~bryan/googleFinalVersionFixed.pdf

How Google Finds Your Needle in the Web's Haystack
http://www.ams.org/samplings/feature-column/fcarc-pagerank

The Google Pagerank Algorithm and How It Works
http://www.cs.princeton.edu/~chazelle/courses/BIB/pagerank.htm