Pattern Recognition and Machine Learning
This is the first text on pattern recognition to present the Bayesian viewpoint, one that has become increasing popular in the last five years. It presents approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible. It provides the first text to use graphical models to describe probability distributions when there are no other books that apply graphical models to machine learning. It is also the first four-color book on pattern recognition. The book is suitable for courses on machine learning, statistics, computer science, signal processing, computer vision, data mining, and bioinformatics. Extensive support is provided for course instructors, including more than 400 exercises, graded according to difficulty. Example solutions for a subset of the exercises are available from the book web site, while solutions for the remainder can be obtained by instructors from the publisher.
Stats for nerds
80recommend
121mentions
104people
2007–2026years
74% recommend25% neutral1% critical
Mentions per year
What Hacker News says
- Understandably I'm becoming a bit dogmatic but I'll say it again, AIMA/PRML/ESL are still the best reference textbooks for foundational AI/ML and will be for a long time.
AIMA is Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig
PRML is Pattern Recognition and Machine Learning by Christopher Bishop.
ESL is Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman.
- PGMs are a great topic to explore (and Koller+Friedman is a great book), but, as a word of caution to anyone interested: implementation of any of these more advanced models remains a major challenge. For anyone building production facing models, even if your problem is a pretty good match for the more interesting PGMs, the engineering requirements alone are a good reason not to go too far down that path.
The PGM book is also structured very clearly for researchers in PGMs. The book is laid out in 3 major section: the models, inference techniques (the bulk of the book), and learning. Which means, if you follow the logic of the book, you basically have to work through 1000+ pages of content before you can actually start running even toy versions of these models. If you do need to get into the nitty-gritty of particular inference algorithms, I don't believe there is another textbook with nearly the level of scope and detail.
Bishop's section on PGMs from Pattern Recognition and Machine Learning is probably a better place to start learning about these more advanced models, and if you become very interested then Koller+Friedman will be an invaluable text.
It's worth noting that the PGM course taught by Koller was one of the initial, and still very excellent, Coursera courses. I'm not sure if it's still free, but it was a nice way to get a deep dive into the topic in a reasonably short time frame (I do remember those homeworks as brutal though!)[0].
0. https://www.coursera.org/specializations/probabilistic-graph...
- I like the axiomatic definition of entropy. Here's the introduction from Pattern Recognition and Machine Learning by C. Bishop (2006):
> The amount of information can be viewed as the ‘degree of surprise’ on learning the value of x. If we are told that a highly improbable event has just occurred, we will have received more information than if we were told that some very likely event has just occurred, and if we knew that the event was certain to happen we would receive no information. Our measure of information content will therefore depend on the probability distribution p(x), and we therefore look for a quantity h(x) that is a monotonic function of the probability p(x) and that expresses the information content. The form of h(·) can be found by noting that if we have two events x and y that are unrelated, then the information gain from observing both of them should be the sum of the information gained from each of them separately, so that h(x, y) = h(x) + h(y). Two unrelated events will be statistically independent and so p(x, y) = p(x)p(y). From these two relationships, it is easily shown that h(x) must be given by the logarithm of p(x) and so we have h(x) = − log2 p(x).
This is the definition of information for a single probabilistic event. The definition of entropy of a random variable follows from this by just taking the expectation.
- Not sure if my experience is relevant, but I did a couple of internships in web dev during my bachelors degree in CS and quickly realized it wasn't for me. I then did a masters and now a PhD in medical imaging where I extensively use machine learning (design and train my own models, doing both supervised and RL) but I wouldn't say I am a researcher in AI/ML.
Because I am still in the academic process, I had the opportunity to take a couple of classes on the subject. Three books that I would recommend going over to make sure your foundation in ML and mathematics are solid are
-Pattern recognition and machine learning by Christopher Bishop
-Mathematics for Machine Learning by Peter Deisenroth
-Deep Learning by Courville, Bengio and Goodfellow
All three are legally available online in some form. I can't say I have any experience in finding a job related to ML though.
- Somewhat ironically, I asked ChatGPT and got a pretty comprehensive response:
Transitioning back into a technical field like AI development after a decade of working in other areas can be challenging but definitely achievable. Here’s a structured plan to help you leverage your existing CUDA and parallel computing knowledge while updating your skills to meet the demands of AI development:
1. Refresh and Update Your Knowledge Since your background is in CUDA and parallel computing, start by refreshing these skills:
CUDA Programming: NVIDIA’s official CUDA documentation and tutorials are great starting points. Utilize resources like CUDA Programming Guide and CUDA Toolkit Documentation.
Parallel Computing: Revisit concepts and explore modern parallel computing frameworks like OpenCL and Thrust.
2. Learn AI and Machine Learning Fundamentals Online Courses: Platforms like Coursera, edX, and Udacity offer excellent AI and ML courses. Key courses include: Machine Learning by Andrew Ng on Coursera. Deep Learning Specialization by Andrew Ng on Coursera. CS231n: Convolutional Neural Networks for Visual Recognition by Stanford (available online for free).
Books: Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Pattern Recognition and Machine Learning by Christopher Bishop. Mathematics for AI: Brush up on linear algebra, calculus, probability, and statistics. Khan Academy and MIT OpenCourseWare can be good resources for this.
3. Gain Practical Experience with AI Frameworks TensorFlow and PyTorch: These are the two most popular frameworks for AI development. Follow official tutorials and documentation: TensorFlow, PyTorch. Engage in projects and Kaggle competitions to apply your knowledge.
4. Dive into GPU-Accelerated Machine Learning NVIDIA’s Deep Learning SDK: Explore NVIDIA's deep learning libraries such as cuDNN, TensorRT, and RAPIDS. Implementations: Try converting existing CPU-based ML models to run on GPUs using your CUDA skills.
5. Build Projects and a Portfolio Personal Projects: Start small by replicating classic ML projects and gradually move to more complex ones. Document your projects on GitHub. Kaggle Competitions: Participate in Kaggle competitions to solve real-world problems, improve your skills, and build a portfolio.
6. Stay Updated and Network Follow AI Research: Regularly read research papers from conferences like NeurIPS, CVPR, and ICML. Websites like arXiv and Papers with Code are useful. Join Communities: Engage with AI communities on platforms like Reddit, Stack Overflow, and specialized forums. Attend local meetups, webinars, and conferences.
7. Consider Advanced Education Professional Certificates: Consider earning AI/ML certificates from universities or companies (e.g., Google, Microsoft). Graduate Programs: If time and resources permit, a postgraduate program in AI/ML could provide structured learning and credibility.
8. Apply for Internships or Junior Roles Internships: Many companies offer internships or junior roles specifically tailored for career switchers. Apply for these to gain hands-on experience. Open Source Contributions: Contributing to open-source AI projects can also provide practical experience and demonstrate your skills to potential employers.
9. Prepare for Job Applications Resume and LinkedIn: Update your resume and LinkedIn profile to reflect your new skills and projects. Mock Interviews: Practice AI/ML interview questions, focusing on coding problems, ML theory, and system design. Summary
Re-entering the AI development field requires a strategic approach: refreshing your existing knowledge, learning new AI/ML concepts, gaining practical experience, and continuously networking. With dedication and structured learning, you can narrow the gap and position yourself for exciting opportunities in AI development within a year.
- Pattern recognition and machine learning by Christopher Bishop.
Related
Neural networks for pattern recognition
3 recommend · 8
2023–2025
Deep Learning
3 recommend · 4
Information Theory, Inference & Learning Algorithms
77 recommend · 111
Pattern classification
8 recommend · 11
The Elements of Statistical Learning
144 recommend · 266
Machine Learning
16 recommend · 29
