Bayesian methods for hackers

What Hacker News says

1. staredSep 2026
   Point taken, but there is a much more fundamental issue with it - and precisely why I wrote "very conservative".

   It is a different problem if we pick two sets from the same data distribution, A and B, and first we have a score on A, then on B. Here we re-run on precisely the same set of Terminal Bench 2.1 problems. It may be that results are so random between runs that each single task has the same probability in a Bernoulli distribution. But more likely, many problems are easy (i.e. each run will solve them consistently), many are too hard (i.e. no run is going to solve them) and only a fraction is somehow in between.

   Maybe there is some good trick to find a proper distribution, but to my knowledge, we would need to run it at least two times on TB2.1 to get any more educated estimates. That said, I am open to new ideas.

   That said, I consider frequentist probability a dirty trick, and that Bayesian is the proper way of doing things (vide David J.C. MacKay" Information Theory, Inference, and Learning Algorithms" and Cam Davidson-Pilon "Probabilistic Programming & Bayesian Methods for Hackers" https://www.inference.org.uk/itprnn/book.pdf, https://dataorigami.net/Probabilistic-Programming-and-Bayesi...).

   Show morethread
2. kianNSep 2025
   I don’t mean for the bar to sound too high. I think working through khan academy’s full probability, calculus and linear algebra courses would give you a strong foundation. I worked through this book having just completed the equivalent courses in college.

   It’s just a relatively dense book. There’s some other really good suggestions in this thread, most of which I’ve heard good things about. If you have a background in programming, I’d suggest Bayesian Methods for Hackers as a really good starting point. But you can also definitely tackle this book head on, and it will be very rewarding.

   Show morethread
3. staredJun 2025
   I think we should declare a moratorium on the use of p-values.

   If you don't understand what a p-value is, you shouldn't use it. If you do understand p-values, you're probably already moving away from them. The Bayesian approach makes much more sense. I highly recommend David MacKay's "Information Theory, Inference and Learning Algorithms," as well as "Bayesian Methods for Hackers": https://github.com/CamDavidsonPilon/Probabilistic-Programmin....

   At the same time, startups aren't science experiments. Our goal is not necessarily to prove conclusively whether something is statistically "better". Rather, our goal is to solve real-world problems.

   Suppose we run an A/B test, and its result indicates B is better according to whatever statistical test we've used. In this scenario, we will likely select B—frankly, regardless of whether B is truly better or merely indistinguishable from A.

   However, what truly matters in practice are the metrics we choose to measure. Picking the wrong metric can lead to incorrect conclusions. For example, suppose our data shows that users spend, on average, two more seconds on our site with the new design (with p < 0.001 or whatever). That might be a positive result—or it could simply mean the new design causes slower loading or more confusion, frustrating users instead of benefiting them.

   Show morethread
4. JabbermonkeyJun 2023
   If you're looking for some Bayesian Stats books that require very little math/stats, I'd recommend starting with these two:

   Think Bayes by Allen Downey:https://allendowney.github.io/ThinkBayes2/

   Bayesian Methods for Hackers by Cam Davidson-Pilon: https://dataorigami.net/Probabilistic-Programming-and-Bayesi...

   They're both an excellent warm up for Statistical Rethinking.

   thread
5. perpetualpatzerJan 2022
   Bayesian Methods For Hackers is a very popular one.

   [0] https://github.com/CamDavidsonPilon/Probabilistic-Programmin...

   thread
6. nknealkJul 2021
   I recommend Bayesian Methods for hackers [1]. It doesn’t go too deep on theory but I feel like it’s well written and has really good coded up examples of theory being applied to problems. It has a relatively narrow scope, but I find myself reaching for methods I learned from the author frequently.

   [1] https://github.com/CamDavidsonPilon/Probabilistic-Programmin...

   thread