Name: Foundations of Data Science
ISBN: 9781108485067

What Hacker News says

renonceMar 2024

I learned the easiest way to do d-dimension sampling in Foundations of Data Science: see https://news.ycombinator.com/item?id=34575637 or https://www.cs.cornell.edu/jeh/book.pdf?file=book.pdf

I don't think it's a good idea to introduce over 20 different methods before talking about the correct one that works for any number of dimensions, say n, and the reason behind its correctness is very obvious:

* Generate a random vector by sampling n standard normal distributions: `vector = np.random.randn(n)`

* Key step: show that the vector has a uniform direction.

The proof is as follows: you look at the probability density function of a normal distribution, which is `p(x)=1/sqrt(2pi)*exp(-x^2/2)`, and the probability density function of the vector is the product of all these densities of its individual dimensions. Now the product `p(x1)p(x2)...p(xn)=1/sqrt(2pi)^n*exp(-(x1^2+x2^2+...+xn^2)/2)` since `exp(a)exp(b)=exp(a+b)` due to power functions.

Now it's easy to see that probability density is invariant to vector length, which means the vector has uniform probability for any specific value of x1^2+x2^2+...+xn^2. Whatever rotation you apply after sampling this vector, since rotation preserves x1^2+x2^2+...+xn^2 by definition, you get the exact same probability density function and therefore the same distribution of vectors.

* Now that the direction of the vector is uniformly sampled, decide the radius separately: for n-sphere the radius is just 1, and for n-ball the volume of the radius is proportional to the nth power of the radius, so you sample a uniform number from [0,1] as the volume and take the nth root as the radius: `radius = np.random.uniform(0, 1)*(1/n)`

* Normalize the vector to the radius: `vector = vector / np.sqrt((vector*2).sum()) * radius`

I think Section 2 of this book provides a much better perspective on the problem of generating uniform random points, since it also provides intuition behind the geometry of high dimensions and properties of the unit ball, etc.

thread

irishspringOct 2020

There are a wealth of such books. For example, elementary stuff[0] and more advanced stuff[1].

[0] Math For Machine Learning by Deisenroth, Faisal, Ong:

https://www.google.com/books/edition/Mathematics_for_Machine...

[1] Foundations of Data Science by Blum, Hopcroft, Kannan:

https://ttic.uchicago.edu/~avrim/book.pdf

thread

psv1Dec 2019

Good free resources:

- MIT: Big Picture of Calculus

- Harvard: Stats 110

- MIT: Matrix Methods in Data Analysis, Signal Processing, and Machine Learning

If any of these seem too difficult - Khan Academy Precalculus (they also have Linear Algebra and Calculus material).

This gives you a math foundation. Some books more specific to ML:

- Foundations of Data Science - Blum et al.

- Elements of Statistical Learning - Hastie et al. The simpler version of this book - Introduction to Statistical Learning - also has a free companion course on Stanford's website.

- Machine Learning: A Probabilistic Perspective - Murphy

That's a lot of material to cover. And at some point you should start experimenting and building things yourself of course. If you'are already familiar with Python, the Data Science Handbook (Jake Vanderplas) is a good guide through the ecosystem of libraries that you would commonly use.

Things I don't recommend - Fast.ai, Goodfellow's Deep Learning Book, Bishop's Pattern Recognition and ML book, Andrew Ng's ML course, Coursera, Udacity, Udemy, Kaggle.

thread

melodramaJul 2019

Great book.

I think it's a good time to mention a couple of nice books (related)

1. Elementary intro to math of machine learning [0]. Its style is a bit less austere than that of OP's. It also has a chapter on probability. It could possible serve as a great prequel to the book linked in the OP.

2. The book on probability related topics of general data science: high-dimensional geometry, random walks, Markov chains, random graphs, various related algorithms etc [1]

3. Support for people who'd like to read books like the one linked in the OP, but never seen any kind of higher math before [2]. This book has a cover that screams trashy book extremely skimpy on actual info (anyone who reads a lot of tech books knows what I am talking about), but surprisingly,it contains everything it says it does and in great detail. Not even actual math textbooks (say, Springer) are usually written with this much detail. Author likes to add bullet point style elaboration to almost every definition and theorem which is (almost) never the case with gazillions of books usually titled "Abstract Algebra", "Real Analysis", "Complex Analysis" etc. Some such books sometimes attach words like "friendly" to their title (say, "Friendly Measure Theory For Idiots") and still do not rise to the occasion. Worse yet, a ton (if not most) of these books are exact clones of each other with different author names attached. The linked book doesn't suffer from any of these problems.

[0] Mathematics For Machine Learning by Deisentoth, Faisal, Ong

https://mml-book.github.io/book/mml-book.pdf

[1] Foundations Of Data Science By Blum, Hopcroft, Kannan

http://www.cs.cornell.edu/jeh/book%20no%20so;utions%20March%...

2] Pure Mathematics for Beginners: A Rigorous Introduction to Logic, Set Theory, Abstract Algebra, Number Theory, Real Analysis, Topology, Complex Analysis, and Linear Algebra by Steve Warner

https://www.amazon.com/Pure-Mathematics-Beginners-Rigorous-I...

thread

moabJul 2017

I overheard some colleagues talking about a recent interview where a candidate with "stellar industry experience", i.e. Kaggle wins and previous ML experience at a valley company, who couldn't explain Bayes rule to them, let alone rederive Naive Bayes. While books like the one below are extremely theoretical, anyone interviewing for these kind of roles should spend at least a week or two just looking through this to see what kind of algorithms and properties are studied in theory.

Foundations of Data Science (Blum, Hopcroft, Kannan) https://www.cs.cornell.edu/jeh/book2016June9.pdf

thread

nilknarfDec 2016

I didn't expect to see Hopcroft/Kannan in that list. I used it in a course taught by Kannan and back then it was called "Computer Science Theory for the Information Age". Apparently the book is now called "Foundations of Data Science". I always thought the old naming was terrible but compared to the new title it actually describes the content a lot better since the book is mainly about CS theory and mathematical foundations for them. It was one of my favorite courses but I would not classify it as ML book.

[1]https://www.cs.cmu.edu/~venkatg/teaching/CStheory-infoage/

[2]https://www.cs.cornell.edu/jeh/book2016June9.pdf

thread