Name: Bernoulli's Fallacy
ISBN: 9780231199940

Bernoulli's Fallacy

There is a logical flaw in the statistical methods used across experimental science. This fault is not a minor academic quibble: it underlies a reproducibility crisis now threatening entire disciplines. In an increasingly statistics-reliant society, this same deeply rooted error shapes decisions in medicine, law, and public policy with profound consequences. The foundation of the problem is a misunderstanding of probability and its role in making inferences from observations. Aubrey Clayton traces the history of how statistics went astray, beginning with the groundbreaking work of the seventeenth-century mathematician Jacob Bernoulli and winding through gambling, astronomy, and genetics. Clayton recounts the feuds among rival schools of statistics, exploring the surprisingly human problems that gave rise to the discipline and the all-too-human shortcomings that derailed it. He highlights how influential nineteenth- and twentieth-century figures developed a statistical methodology they claimed was purely objective in order to silence critics of their political agendas, including eugenics. Clayton provides a clear account of the mathematics and logic of probability, conveying complex concepts accessibly for readers interested in the statistical methods that frame our understanding of the world. He contends that we need to take a Bayesian approach—that is, to incorporate prior knowledge when reasoning with incomplete information—in order to resolve the crisis.

What Hacker News says

perrygeoNov 2024

Origin - A genetic history of the Americas by Jennifer Raff. Recent evidence of how humans came to the Americas; it's a significantly different story than our conventional wisdom account.

Cows, Pigs, Wars, and Witches by Marvin Harris. How cultural norms establish collective behavior.

The Extermination of the American Bison. Written in the late 1800s as the tragedy was unfolding. Lots on great detail on how and why we decimated a keystone species in a matter of a few decades.

Plato's Revenge by William Ophuls. A concise manifesto on societal change. Lays out a vision for a new post-growth, post-oil, ecological consciousness that incorporates modern science, religion and philosophy.

Four Thousand Weeks. Far from a self-help time-management book, this is more a philosophy book on human's relationship with time. Delightful read.

Breath by James Nestor. Modern humans are breathing incorrectly due to factors of modern life. An exploration of the science and art of breathing properly.

Bernoulli's Fallacy by Aubrey Clayton. In the statistical wars, this book takes a strong stance: Bayesian is unequivocally better. He exploses the racist/colonial history of frequentist statistics, as well as the fundamental flaws in their math. Posits that mis-applied frequentist methods are to blame for the reproducibility crisis in science.

How the World Really Works by Vaclav Smil. A no-nonsense, factual account of the systems that keep our modern globalized society running.

Elements of Clojure by Zach Tellman. A concise, high-level philosophy about how we build and compose abstractions. One of those books where every sentence is gold. My only wish that the book wasn't tied to Clojure (most of the book is applicable to any language, despite the title).

Tidy First? by Kent Beck. How and when to make structural changes (refactoring, architecture, paying down tech debt) vs behavioral changes (new features, bug fixes). You need both, and you can use economic theory as a guide for when to invest in one or the other. But you need to treat them differently.

Numerical Linear Algebra for Programmers by Dragan Djuric. A code-centric introduction to common linear algrebra routines, Using Clojure and the neaderthal library - a high-level API for fast math optimized for CPU and GPU hardware.

thread

rasseJan 2024

Somewhat related is the "boy or girl paradox" ( https://en.m.wikipedia.org/wiki/Boy_or_girl_paradox ), where the answer to "Mr. Smith has two children. At least one of them is a boy. What is the probability that both children are boys?” may be 2/3 instead of 1/2. The problem is much debated.

I was made aware of this paradox by the book Bernoulli's Fallacy by Aubrey Clayton ( https://www.goodreads.com/book/show/55825328-bernoulli-s-fal... ), which I do recommend as an interesting read.

thread

jmuncasterDec 2023

Bernoulli’s Fallacy by Aubrey Clayton is a good treatment of this subject and where frequentist statistics fails.

thread

JHonakerJun 2022

I'm a statistician, and I have to say, I agree with the fundamental premise of the author:

The problems with null hypothesis significant testing (NHST) are a direct result of two things:

1. the false air of objectivity it provides

2. the lack of consideration of the relative likelihood of the data under any alternative

I also agree that Pearson, Fisher, and Neyman knew what they were doing when they appealed to these methods' "objectivity" to support their terrible views.

I also find it just absolutely hilarious that Fisher's adamant opposition to Bayesian inference was the (in his opinion) absurdity of the uniform prior distribution, and yet one of his most famous contributions (Maximum likelihood estimation) is essentially the same result you'd get from the Bayesian estimate of the most likely parameter with a uniform prior.

If anyone is interested in the history of this stuff along with a discussion of the philosophical meaning of probability from the same author, his book Bernoulli's Fallacy is a pretty good read. My only qualm with it is the lack of much exposure on how to actually do Bayesian inference or a discussion on what it's like to do it in a modern setting. There's a brief few pages at the very end that mention things like Markov Chain Monte Carlo and the fact that most modern Bayesian inference is necessarily numerical approximations to the posterior, not the analytical solution (but that's not necessarily a bad thing nor is it an escape-able problem).

If you want a book that makes many of the same philosophical arguments about probability but also backs it up with math and descriptions on how to do Bayesian inference, Edwin Jaynes's Probability Theory: The Logic of Science is for you. It's quite literally a modern cult classic. People that read this book tend to go out and evangelize for it.

My final comment is that there is a place for frequentist methods, even if you're a modern Bayesian. The theory of Markov Chain Monte Carlo (one of the key methods in making modern Bayesian inference work) is exactly the domain of frequentist probability: Long runs of repeated things. Many of the diagnostic tools for assessing whether the inferences of MCMC results can be trusted are hinged on results from the frequentist interpretation and are steeped in their strange-ness. For instance, the lack of an issue is taken lack of evidence of an issue rather than evidence of no issue. The interconnected-ness of the two is unfortunately missing from the author's work (that I have seen).

thread

woolionAug 2024

Small unrelated bug report: in your "Book review: Bernoulli's Fallacy" article, there is a link to: https://www.thatgeoguy.ca/blog/%7B%7B%20site.baseurl%20%7D%7...

I don't have much time to read through all this now but I'll check later, looks like great write-ups!

thread

perrygeoMar 2024

Relearning statistics with a Bayesian approach. My undergrad education was in social science research methods and I spent 4 years learning the strict frequentist approach of orthodox statistics. It made sense for highly-controlled experiments and simple dice games. But it broke down horribly when faced with any complexity and I never understood why. 25 years later, it's time to fill in the gaps. My reading list:

- E.T. Janes "Probability Theory: The Logic of Science" provides the fundamental theory.

- Robert McElreath "Rethinking Statistics" provides a practical application of the theory in R.

- Andrew Clayton's "Bernoulli's Fallacy" is a non-technical book that provides historical context to the frequentist vs bayesian debate.

I'm fairly convinced now that Bayesian approaches have more mathematical rigor than the crusty old heuristics of traditional statistics. But in terms of user-experience, doing Bayesian calculations still requires more effort on model design and more compute power. It's flexible to a fault, without a well-defined workflow. There is a strong temptation to follow the easy path - shove your data into a black box and publish if p<0.05. It's going to take a generation of (re)training and improvements to statistical software before Bayesian methods are widespread.

thread