The Book of Why
A Turing Award-winning computer scientist and statistician shows how understanding causality has revolutionized science and will revolutionize artificial intelligence “Correlation is not causation.” This mantra, chanted by scientists for more than a century, has led to a virtual prohibition on causal talk. Today, that taboo is dead. The causal revolution, instigated by Judea Pearl and his colleagues, has cut through a century of confusion and established causality–the study of cause and effect–on a firm scientific basis. His work explains how we can know easy things, like whether it was rain or a sprinkler that made a sidewalk wet; and how to answer hard questions, like whether a drug cured an illness. Pearl’s work enables us to know not just whether one thing causes another: it lets us explore the world that is and the worlds that could have been. It shows us the essence of human thought and key to artificial intelligence. Anyone who wants to understand either needs The Book of Why.
Stats for nerds
59recommend
144mentions
103people
2018–2026years
55% recommend40% neutral5% critical
Mentions per year
What Hacker News says
- Would love to see a causal model [0] to help better understand all of the mediators considered as well as confounders. I'm close to finishing up an interesting read from Judea Pearl/Dana Mackenzie - The Book of Why: The New Science of Cause and Effect [1]. Talks alot about Causal Models, Causal Inference, the 3 ladders of causation, etc. I liked the graphical approach to help outline exactly how one thinks about direct and indirect effects and how it facilitates counterfactual analysis and causal mediation analysis.
- It becomes interesting. You sounds as an advent of Church of Science from the very beginning, but now it becomes unmistakable. Science has no monopoly on truth. I'm not familiar with academy, you are right, I don't know how much Science share your ideas, but if it is, then Science has defeated itself. I hope that it is not so, and I believe it is not so.
Below there is a list of reasons why we cannot grant Science a monopoly on truth. I'm not claiming it is a comprehensive list, it is just reasons from top of my mind. But before that I'll say two more things.
Science doesn't have a monopoly on truth and Science is not a perfect instrument. Therefore I'll seek wisdom in all places, and I'll use my own brains to think, even if I'm not an expert in a topic. I will not ask medium, but it is not because I believe it is absolutely useless, it is because I have a limited time and I need to prioritize sources with high expected value of their answers. But still I will not bind myself to highest value sources either, because the path to the truth is not linear, you'd better remember about Monte Carlo methods and add some randomness to your moves.
I want also to say about "unanswered questions". It is really good, that you feel yourself comfortable despite uncertainty. Most of people are not, they can't sleep if they have unanswered questions and they could pick a random answer just to make the question answered. But it is possible to do better, than just stuff a question in a pile of unanswered questions. If you accept the idea of an answer as a probability distribution over all possible answers, then you can have an answer for any question and move all the uncertainty into the distribution of the answer. You can deal with the uncertainty in a more conscious way, and sometimes using more intricate methods. You can seek answer, find no definitive "yes" or "no", but still update your distribution. The unanswered questions stop being unanswered, they become questions with high entropy answers. Entropy is a continuous function, so question can become more unanswered or less unanswered.
And now the list of reasons to reject Science claim for a monopoly on truth.
1. There are phenomena Science can't deal with due to its methods. The most glaring class of such phenomena are single irreproducible events. Did Christ rise from death? Science can't answer this question and probably it never won't be able to answer it, because it can't do multiple observations. Scott Alexander recently wrote again on topic of a collective vision of Virgin Mary[1], and science can't tackle the question. I think you should read it through. Because you'll notice that it seems that now science can meaningfully join the discussion, because there are similar events, but before these events we found, all the science could do about the vision, is to keep its proud silence on the topic.
2. We cannot trust Scott Alexander, but we cannot trust Science either. Science can be mistaken, and sometimes it may be mistaken for no good reason at all. Did you read Judea Pearl "The Book of Why"? Pearl tells the history of statistics in his book, and particularly the roles of Pearson and Fisher, who doesn't look as people prioritizing truth over everything else, they where fighting the truth due to... well, I'd say because they were humans with all our failings. Science can be led astray in its search for truth behind Alzheimer disease because of a junk-paper published 30 years ago. Or Science could spend enormous efforts to build String Theory and then... just chuck it away.
We should hope that Science will correct itself eventually in all such cases, but should we just wait for it, or maybe we are allowed to stray from Science and use our brains instead? Maybe Science is close to truth than anything/anyone else, but still it is not the Truth. Which means than you need to use your own brains, and to accept any statement with a grain of salt. We have to part with boolean logic and embrace the probabilistic nature of truth even when we talk about scientific truths.
3. Science develops the scientific method and employs the state of art methods to seek truth, but it is a scientific propaganda. xD I mentioned The Book of Why of Judea Pearl, which shows examples of Science rejecting better methods, and the causality proposed by Judea Pearl and his students is not a method science really wield. Bayesianism is the previous idea of Judea Pearl, it is older than causality by 10 years, and Science still relies on debunked p-values[2]. There are better methods, but Science still clings to old ones, rejecting the prospect of learning new tricks.
4. Science is a part of a bigger system and it can't avoid pressure from it. SJW pressured Science to a point, when its answers on questions including words "black", "white", "male", "female" are just plainly unreliable. Scientists can't publish a paper stating that black people are somehow "worse" than white. It doesn't matter if the paper contains truth or not, scientist just not allowed to publish it. So now any study talking about correlation of skin color or sex or gender with anything is just don't worth time needed to read it. I don't know other examples of this, but you need to be careful when seeking scientific answers, and keep in mind that some answers cannot be uttered. Or science is not allowed to diagnose politicians based on observations of their behavior. Science is restricted by society, culture, moral and ethical considerations. We should keep it in mind when we are seeking scientific answers.
5. There is knowledge Science never bothered itself to incorporate, or it just can't incorporate it. If my granny shows me how to use a specific herb to treat a wound, should I reject her knowledge as unscientific? Lets suppose I can't find a scientific answers about the herb, so what should I do? But knowledge can be just beyond Science. Science works with knowledge that can be written on a paper. If it can't then Science is out of luck. There is intuition for example, it cannot be written down. I use it to write English, I don't know grammar rules, I rely on intuition. I can't formalize my intuition, I can just use it. Well... English maybe not the best example for me, because English is my second language, and I'm not enough proficient with it to compete with professional linguists, but I am proficient in Russian to use it in ways that no linguist could explain. No grammar would accept my ways, but other Russians will understand what I'm doing. You see? I'm more proficient in Russian than Science with all its linguistic studies and thousands of linguists.
6. There are questions that are beyond Science. Like what does it mean to be conscious? When scientists try to answer this question, they are not better than laymans. For example, Anil Seth says that "consciousness is biological"[3] and kinda stops there. What a shame. Or Steven Hawking proclaiming that philosophy is dead. You can't just ask Science what is conscious and get an answer, because Science doesn't understand the question: it is a philosophical question right now. It can become a scientific question, but for now it is not.
[1] https://www.astralcodexten.com/p/a-buddhist-sun-miracle [2] https://figshare.le.ac.uk/articles/journal_contribution/Agai... [3] https://www.noemamag.com/the-mythology-of-conscious-ai/
- Causal has a specific meaning related to causal modeling, most studies can't show causality, a lot only show correlation[1].
And the third one seems to be about effect sizes. But a lot of this is still concerning, even if they appear to be trying to say technically true but misleading things.
[1] Yes, newer methods can show causation, not just correlation. See The Book of Why, by Judea Pearl for an introduction to how that works.
- There are some interesting examples in the "The Book of Why" [y] by Pearl and Mackenzie. This book is all about causal inference & has examples from things like drug trials where the analysis and inferences drawn from analysing experimental data depend upon what causal model you assume is generating the data, and if you pick an analysis that implicitly assumes the wrong underlying causal structure then your inferences drawn from the data may be wrong. There's a chapter about the historical scientific debate around if smoking causes lung cancer.
Andrew Gelman has an interesting review of the book [g], from the perspective of someone working within the statistical establishment that Pearl's book often critiques.
[y] https://www.hachettebookgroup.com/titles/judea-pearl/the-boo...
[g] https://statmodeling.stat.columbia.edu/2019/01/08/book-pearl...
- Unfortunately, this is an observational study and when you get to the confounding part, they kind of shrug their shoulders and say “well, we included a bunch of covariates that should reduce make the bias go away”, but there’s no causal diagram so we have no idea how they reasoned about this. If you’ve read even something layman friendly like Pearl’s Book of Why you should be feeling nervous about this.
- Immunity to Change by Robert Kegan
The Book of Why by Judea Pearl
The Idea Factory by Jon Gertner
7 Seconds to Die by John Antal
Related
Causality
17 recommend · 37
Causal Inference in Statistics
5 recommend · 9
AI Ethics
1 recommend · 2
Statistical Rethinking
81 recommend · 113
Data Science
4 recommend · 7
The enigma of reason
8 recommend · 15
This Changes Everything
9 recommend · 18
