Understanding Machine Learning
"Machine learning is one of the fastest growing areas of computer science, with far-reaching applications. The aim of this textbook is to introduce machine learning, and the algorithmic paradigms it offers, in a principled way. The book provides an extensive theoretical account of the fundamental ideas underlying machine learning and the mathematical derivations that transform these principles into practical algorithms. Following a presentation of the basics of the field, the book covers a wide array of central topics that have not been addressed by previous textbooks. These include a discussion of the computational complexity of learning and the concepts of convexity and stability; important algorithmic paradigms including stochastic gradient descent, neural networks, and structured output learning; and emerging theoretical concepts such as the PAC-Bayes approach and compression-based bounds. Designed for an advanced undergraduate or beginning graduate course, the text makes the fundamentals and algorithms of machine learning accessible to students and non-expert readers in statistics, computer science, mathematics, and engineering"
Stats for nerds
12recommend
18mentions
16people
2014–2023years
72% recommend28% neutral
Mentions per year
What Hacker News says
- I have not seen mentioned so far in this thread the following book, which I can't recommend more highly:
Understanding Machine Learning: From Theory To Algorithms – Shai Shalev-Shwartz
- I like to explain that AI > ML > NN.
Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving.
If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gentle intro would be Learning From Data [2]. If someone recommends a book/paper it would be awesome, I'm always on the look.
[1] https://www.cs.huji.ac.il/w~shais/UnderstandingMachineLearni... [2] https://work.caltech.edu/telecourse
- If you are looking to have a mathematical understanding of the field then I really enjoyed "Understanding Machine Learning: From Theory to Algorithms" by Shalev-Shwartz and Ben-David.
- I really enjoyed Ben-David (and Shalev-Schwartz)'s book "Understanding machine learning". It's essentially about theory though, not "learn all about machine learning in torchsorflow in 10 days".
- I typically recommend a few different books to everyone who finishes the bootcamp, based on a self-assessment they take. I recommend some books based on their strengths, so they can find a career path sooner, and some books based on their weaknesses, so they can widen their cone of oppportunity within ML.
In our consultancy, data science is done in Python and SQL (and PySpark, but I don't hand out books on that during bootcamp!), and ML delivery is a combination of math, software engineering, and architecture/product owner disciplines.
If you're strong in software engineering, I recommend Machine Learning Mastery with Python by Jason Brownlee as it's very hands-on in Python and helps you run code to "see" how ML works.
If you're weak in software engineering and Python, I recommend A Whirlwind Tour Of Python by Jake VanderPlas, and its companion book Python Data Science Handbook.
If you're strong in architecting / product management, I recommend Building Machine Learning Powered Applications by Emmanuel Ameisen since it explains it more from an SDLC perspective, including things like scoping, design, development, testing, general software engineering best practices, collaboration, etc.
If you're weak in architecting / product management, I typically recommend User Story Mapping by Jeff Patton and Making Things Happen by Scott Berkun, which are both excellent how-tos with great examples to build on.
If you're strong in math, I recommend Understanding Machine Learning from Theory to Algorithm by Shalev-Shwartz and Ben-David, as it has all the mathematics for ML and actually has some pseudocode for implementation which helps bridge the gap into actual software development (the book's title is very accurate!)
For someone who is weak in the math of ML, I recommend Introduction to Statistical Learning by Hastie et al (along with the Python port of the code https://github.com/emredjan/ISL-python ) which I think does just enough hand holding to move someone from "did high school math 20 years ago" to "I understand what these hyperparameters are optimizing for."
Anyway, I've spent a lot of time reading and reviewing books about ML, and my key takeaway is ones that get you closer to writing actual code to solving actual problems for actual people are the ones to focus on.
- I could rant about this for hours. I actually just went to a defense for a deep learning paper that had a ton of abstract algebra. I am honestly not really a fan of deep learning and algebra because all the papers to me like- seem to stop at describing some really basic feedforward network as some really specific mathematical structure but these theories a) provide very little explanation of empirical phenomena b) provide no new directions of research in terms of like useful network architectures.
I haven't really come across algebra in machine learning other than people applying it to deep learning.
i.e. https://arxiv.org/pdf/1802.03690.pdf
ie. https://icml.cc/Conferences/2018/Schedule?showEvent=2048
I don't personally find papers like this valuable but idk I have never really enjoyed abstract algebra.
For areas of mathematics to do theory in ML (and to do ML more generally!)
-probability/concentration/hoeffding bounds [the PAC model] [Key]
-linear algebra [key]
-optimization [key]
for books
-understanding machine learning by shai ben david
This book is nice since it really balances theory with a more practical understanding.
-An Introduction to Computational Learning Theory by kearns is a classic [low priority]. this is fun since the proofs are simple and deep but is very very far away from practical algorithms.
-convex optimization by boyd
Course Notes:
[I think a good alternative to blogs is stalking course notes for other schools-they are very often public.]
- http://ttic.uchicago.edu/~avrim/MLT18/index.html
good learning theory course by avrim blum who is a big deal in learning theory and theory.
- tim roughgardens notes are a blessing for algorithms and theory [seriously he should have a patreon or something]
https://theory.stanford.edu/~tim/notes.html
Blogs:
this is ben recht's blog and is filled with ML wisdom.
-https://blogs.princeton.edu/imabandit/ not quite learning theory but a lot of ML adjacent stuff
I don't read many blogs as I should tbh so other people can give better advice
VIDEOS https://www.youtube.com/channel/UCW1C2xOfXsIzPgjXyuhkw9g
This is the simons institute youtube channel. probably the best single location for recordings of TALKS in computer science-good amount of ML talks.
Related
Foundations of machine learning
4 recommend · 9
2020–2021
Building Machine Learning Powered Applications
3 recommend · 4
Learning From Data
31 recommend · 49
Introduction to Statistical Learning
85 recommend · 120
Machine Learning
16 recommend · 29
Information Theory, Inference & Learning Algorithms
77 recommend · 111
