Foundations of machine learning

What Hacker News says

1. stochastic_monkApr 2018
   I recommend Shai Shalev-Schwartz and Shai Ben-David's Understanding Machine Learning: From Theory to Algorithms [0]. I've also used and found Mohri's Foundations of Machine Learning quite insightful [1]. Usually, between the two books, at least one's proof is easy to follow.

   Get both unless you're only getting one, in which case, get Shai Shalev-Schwartz's.

   [0] http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning...

   [1] https://mitpress.mit.edu/books/foundations-machine-learning

   Show morethread
2. stochastic_monkNov 2017
   Convex: Bertsekas - Convex Optimization Theory, Convex Optimization Algorithms. Nesterov - Lecture Notes (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.693...)

   Statistical/Theoretical: Shai Shalev-Schwartz & Shai Ben-David's Understanding Machine Learning (http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning...) Mohri's Foundations of Machine Learning (https://cs.nyu.edu/~mohri/mlbook/)

   The two above courses could share SSS's Online Learning text (https://www.cs.huji.ac.il/~shais/papers/OLsurvey.pdf). To be fair, the stochastic variants of most optimization algorithms can be learned reasonably quickly off of a statistical machine learning/basic optimization background. There's the option of Spall's Intro to Stochastic Search and Optimization, which covers neural networks, reinforcement learning, annealing, MCMC, and a wide variety of other applicaitons and techniques. (http://www.jhuapl.edu/ISSO/)

   Similar to what kxyvr said, I also don't know of any killer linear algebra text, which is why I think a course is so useful. The matrix cookbook is helpful along the way. kxyvr is also entirely right that general nonlinear optimization is important -- though perhaps less indispensable. (Going the other way, the Bertsimas linear optimization textbook I've had for years mostly gathers dust.)

   For PGMs: I got Predicting Structured Data back when it was new (https://mitpress.mit.edu/books/predicting-structured-data), but I think that Chris Bishop's treatment in PRML is easier to follow. He has some lecture slides which expand on it quite well. (https://www.microsoft.com/en-us/research/people/cmbishop/)

   Bishop would also be my go-to intro ML book over Murphy.

   I can't in fairness offer recommendations for the rest of the intermediate undergraduate math texts because I took them so long ago, but I can say that I have benefited from reviewing the MIT OCW courses from time to time.

   Show morethread
3. lindberghDec 2016
   Also consider Foundations of Machine Learning (2012) by Meryar Mohri. It is a very good book with a statistical learning approach to ML. If you're interested by statistical guarantees to ML algorithms, eg. why the SVM algorithm works and its sample complexity (the expected out of sample error in function of the sample size), it's the book you need.
   thread
4. zintinio5Apr 2016
   * Foundations of Machine Learning

   * All of Statistics

   * Doing Bayesian Data Analysis

   Also the ML specialization on Coursera

   thread
5. kashifrJul 2014
   Form my own journey I would say that a good place to start for graphical models might be "Bayesian Reasoning and Machine Learning" by Barber. It's free (http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...). I haven't read through it, but I've heard good things. However, it doesn't cover some basic things like SVM, RVM, Neural Networks...

   For those I'd suggest "Pattern Recognition and Machine Learning" by Bishop. I've read throughout this and it's really well organized and thought out. For more mathematically advanced ML stuff I'd suggest "Foundations of Machine Learning" by Mohri. For a good reference for anything else I'd suggest "Machine Learning: A Probabilistic Perspective" by Murphy. For more depth on graphical models look at "Probabilistic Graphical Models: Principles and Techniques" by Koller.

   On the NLP front there's the standard texts "Speech and Language Processing" by Jurafsky and "Foundations of Statistical Natural Language Processing" by Manning.

   I also like "An Introduction to Statistical Learning" by James, Witten, Hastie and Tibshirani.

   Show morethread
6. miltavaMar 2025
   This looks interesting. Does anyone knows how it compares to other learning theory books like foundations of machine learning [1] in terms of depth and approachability?

   [1] https://cs.nyu.edu/~mohri/mlbook/

   thread