Probabilistic Graphical Models
A general framework for constructing and using probabilistic models of complex systems that would enable a computer to use available information for making decisions. Most tasks require a person or an automated system to reason—to reach conclusions based on available information. The framework of probabilistic graphical models, presented in this book, provides a general approach for this task. The approach is model-based, allowing interpretable models to be constructed and then manipulated by reasoning algorithms. These models can also be learned automatically from data, allowing the approach to be used in cases where manually constructing a model is difficult or even impossible. Because uncertainty is an inescapable aspect of most real-world applications, the book focuses on probabilistic models, which make the uncertainty explicit and provide models that are more faithful to reality. Probabilistic Graphical Models discusses a variety of models, spanning Bayesian networks, undirected Markov networks, discrete and continuous models, and extensions to deal with dynamical systems and relational data. For each class of models, the text describes the three fundamental cornerstones: representation, inference, and learning, presenting both basic concepts and advanced techniques. Finally, the book considers the use of the proposed framework for causal reasoning and decision making under uncertainty.
Stats for nerds
16recommend
42mentions
37people
2010–2025years
40% recommend55% neutral5% critical
Mentions per year
What Hacker News says
- I used to take a lot of courses on cousera and EdX. I still take some here and there, but not as many as before. Some of the courses are amazing and unbelievably rewarding, like Daphne Koller's Probabilistic Graphical Models, Robert Sedgewick's Analytical Combinatorics, and Gerald Sussman's course on system optimizations. I'm very grateful for such learning opportunities. Unfortunately over time, I also found that these courses had diminishing returns for the following reasons:
- Due to the nature of MOOC, the assignments are largely either multiple-choice questions or programming assignments that merely asked students to fill in some blanks in some functions (there are a few exceptions of course). What a descent US university does really well is challenging students with tough yet insightful and inspiring assignments. That's how students learn deeply and retain the knowledge, at least for me. Merely listening to lectures and ticking off a few ABCDs hardly helps real learning.
- Lack of feedback. A university course assigns TAs, gives tutorials and office hours, grades assignments with detailed feedbacks, and it is so much easier to form study groups and have high bandwidth discussions. MOOCs try their best to offer such help, but they don't work as well or at least not as conveniently.
- Many courses are watered down. For instance, Andrew Ng's ML course on Coursera is far less rigorous than that (229? I forgot) offered in Stanford? The course is great for students to gain some intuition, but I'm not sure if it's good enough for one to build solid ML foundations.
- Probabilistic Graphical Models by Koller & Friedman. In anything statistics and ML related, being able to deal with complicated probabilitistic things that are all related is really useful. This book gives you that toolkit. It's a "strong foundations" kind of book, rather than a bunch of methods you'll use directly.
- > study textbooks. Do exercises. Treat it like academic studying
This. Highly recommend Russel & Norvig [1] for high-level intuition and motivation. Then Bishop's "Pattern Recognition and Machine Learning" [2] and Koller's PGM book [3] for the fundamentals.
Avoid MOOCs, but there are useful lecture videos, e.g. Hugo Larochelle on belief propagation [4].
FWIW this is coming from a mechanical engineer by training, but self-taught programmer and AI researcher. I've been working in industry as an AI research engineer for ~6 years.
[1] https://www.amazon.com/Artificial-Intelligence-Modern-Approa...
[2] https://www.amazon.com/Pattern-Recognition-Learning-Informat...
[3] https://www.amazon.com/Probabilistic-Graphical-Models-Princi...
- I recommend David MacKay's Intro to Info Theory class: https://www.youtube.com/watch?v=BCiZc0n6COY
There is a free book for it at: http://www.inference.org.uk/mackay/itila/
Daphne Koller's 3 coursara classes on Probabilistic Graphical Models are also really good.
- Here are the resources I found useful: ========================================== Advices from Open AI, Facebook AI leaders
Courses You MUST Take:
Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-learning) /// Class notes: (http://holehouse.org/mlclass/index.html)
Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.
(https://work.caltech.edu/telecourse.html)
Neural Networks and Deep Learning (Recommended by Google Brain Team) (http://neuralnetworksanddeeplearning.com/)
Probabilistic Graphical Models (https://www.coursera.org/learn/probabilistic-graphical-model...)
Computational Neuroscience (https://www.coursera.org/learn/computational-neuroscience)
Statistical Machine Learning (http://www.stat.cmu.edu/~larry/=sml/)
From Open AI CEO Greg Brockman on Quora
Deep Learning Book (http://www.deeplearningbook.org/) ( Also Recommended by Google Brain Team )
It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg
2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (http://ee263.stanford.edu/) or Linear Algebra (MIT)
(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...)
Neural Networks for Machine Learning — Geoff Hinton (Coursera) https://www.coursera.org/learn/neural-networks
Neural Nets — Andrej Karpathy’s CS231N (Stanford)
Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley)
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/
Deep RL — John Schulman’s CS294–112 (Berkeley) http://rll.berkeley.edu/deeprlcourse/
- Yes, I did my research but there is no such interactive tutorial online like Treehouse or Codecademy. There are so many tutorials but none of it tells you the whole path.
Here are the resources I found useful:
========================================== Advices from Open AI, Facebook AI leaders
Courses You MUST Take: Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-learning) /// Class notes: (http://holehouse.org/mlclass/index.html)
Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.(https://work.caltech.edu/telecourse.html)
Neural Networks and Deep Learning (Recommended by Google Brain Team) (http://neuralnetworksanddeeplearning.com/)
Probabilistic Graphical Models (https://www.coursera.org/learn/probabilistic-graphical-model...)
Computational Neuroscience (https://www.coursera.org/learn/computational-neuroscience)
Statistical Machine Learning (http://www.stat.cmu.edu/~larry/=sml/)
From Open AI CEO Greg Brockman on Quora
Deep Learning Book (http://www.deeplearningbook.org/) ( Also Recommended by Google Brain Team )
It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg
2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (http://ee263.stanford.edu/) or Linear Algebra (MIT)(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...)
Neural Networks for Machine Learning — Geoff Hinton (Coursera) https://www.coursera.org/learn/neural-networks
Neural Nets — Andrej Karpathy’s CS231N (Stanford) http://cs231n.stanford.edu/
Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley) https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/
Deep RL — John Schulman’s CS294–112 (Berkeley) http://rll.berkeley.edu/deeprlcourse/
From Director of AI Research at Facebook and Professor at NYU Yann LeCun on Quora
In any case, take Calc I, Calc II, Calc III, Linear Algebra, Probability and Statistics, and as many physics courses as you can. But make sure you learn to program.
Related
Bayesian reasoning and machine learning
15 recommend · 22
Pattern Recognition and Machine Learning
80 recommend · 121
The Elements of Statistical Learning
144 recommend · 266
Doing Bayesian Data Analysis
22 recommend · 33
Introduction to Statistical Learning
85 recommend · 120
Information Theory, Inference & Learning Algorithms
77 recommend · 111
