Willful Ignorance

What Hacker News says

1. kqrSep 2022
   = Out of the Crisis by Deming.

   Everyone ought to read and absorb Deming. It's hands-down the most life-changing read I've ever come across, by a wide margin.

   Dealing with life statistically rather than overreacting to randomness means it's possible to achieve much higher levels of effeciveness and productivity. Working with people to unleash their inner motivation is vastly superior to trying to carrot-and-whip them around.

   Understanding that a specific process will yield a specific outcome, whatever outcome you wish for, is how to avoid getting the same result over and over while thinking you ought to see something different because gosh-darned if you aren't trying really hard!!

   = Willful Ignorance by Weisberg.

   This isn't that good a book, in that it doesn't cover very much content at all, but it really gave me a good intuition for what statistics and probability is about that has helped me understand more advanced things than I was able to before.

   Both this book and Deming will take a few years to digest. You'll want to practise it in real life, then come back to read it again, then practise some more. Eventually the lessons will start to stick.

   = Performance Modeling and Design of Computer Systems by Mor Halchor-Balter.

   Despite the name, this is a book about queueing theory. You know, that branch of statistics that shows up literally everywhere, except nobody seems to understand that it does. I have been dabbling in queueing theory before this book, but the focus this book put on practical applications really made me understand it on a deeper level than before.

   = Working Effectively with Legacy Code by Michael Feathers.

   I didn't originally want to put this one in this list because it's so predictable, but I can't not. I keep coming back to it and re-reading portions of it every time I have to deal with tricky code. It's just that good at explaining clever ways to get things under test, so they can be worked on with confidence.

   = How to Measure Anything by Douglas Hubbard.

   What I really take away from this is how powerful it is to be able to estimate ranges that are true 90 % of the time. It's hard to convince others of what a superpower this is, but I hope to work with people who understand that some day.

   Show morethread
2. jtsukenJun 2021
   Here is a list of books for those wishing to dig deeper:

   1) The Information by Gleick is a book covering everything from biographies of the key historical figures and contributors to contemporary applications of information theory in quantum mechanics.

   2) Elements of Information Theory by Joy Thomas and Thomas Cover is a thorough and engaging textbook.

   3) Quantum information and quantum computation by Nielsen and Chuang explaining both classical and quantum information with some practical examples.

   4) Entropy and Information by Volkenstein a Soviet popular science book with math-heavy examples from biology, chemistry and physics

   and as an aside I can also recommend "Willful Ignorance: The Mismeasure of Uncertainty" by Weisberg, which is an engaging history review of probability theory and its key inventors

   Show morethread
3. kqrApr 2026
   That's not how pre-statistical reasoning works. We have known for a long time that coins tend to land on either side around half the time. But before statistics, the outcome of any individual coin toss was considered "not uncertain, merely unknown".

   Before you toss the coin, God has determined with full certainty on which side it will land based on everything riding on that coin toss and all the third-order consequences, in His infinite wisdom. It cannot land on any side other than the preordained. The way you find God's will is to flip the coin.

   To the pre-statistical brain it was unthinkable (and probably blasphemeous) to perform any sort of expected value calculation on this.

   We know today that the frequency is useful for making decisions around the individual throws. Back then, that connection just wasn't there. Each throw was considered its own unique event.

   (We can still see this in e.g. statistically illiterate fans of football. Penalty kicks are a relatively stable random process -- basically a weighted coin toss. Yet you'll see fans claim each penalty kick is a unique event completely disconnected from the long-run frequency.)

   Statistics is a very young invention. As far as we know, it didn't exist in meaningful form anywhere on Earth until the 1600s. (However, if it existed in the Americas earlier than that, that would explain why it suddenly popped up in Europe in the 1600s...)

   ----

   Important edit: What I know about this comes mostly from Weisberg's Willful Ignorance as well as A World of Chance by Brenner, Brenner, and Brown. These authors' research is based mostly on European written sources, meaning the emphasis is on how Europeans used to think about this.

   It's possible different conceptualisations of probability existed elsewhere. It's possible even fully-fledged statistical reasoning existed, although it seems unlikely because it is the sort of thing that relies heavily on written records, and those would come up in research. But it's possible! That's what I meant by the last parenthetical – maybe Europeans didn't invent it at all, but were merely inspired by existing American practice.

   Show morethread
4. kqrAug 2024
   I think I got most of this from Willful Ignorance (Weisberg, 2014) but it was a while since I read it (and this was before I made more detailed notes) so I might be misremembering.

   There are what looks like fantastic books on the early days of probability and statistics aside from Weisberg (Ian Hacking is an author that comes to mind, and maybe Stephen Stigler?) but I have not yet taken the time to read them – I'll read more and write something up on this some day.

   But statistical understanding is such a recent phenomenon that we have been able to document its rise fairly well[1] which is fascinating in and of itself.

   [1]: And yet we don't know basic stuff like when the arithmetic mean became an acceptable way to aggregate data. Situations that obviously (to our eyes) call for an arithmetic mean in the 1630s were resolved some other way (often midrange, mode, or just a hand-picked observation that seemed somehow representative) and then suddenly in the 1680s we have this letter to a newspaper which uses the arithmetic mean as if it was the obvious thing to do and then its usage is relatively well documented after that. From what I understand we don't know what happened between 1630 and 1680 to cause this change in acceptance!

   Show morethread
5. chmsky00Oct 2021
   It would be nice if the era of economic theory were to end.

   Herbert Weisberg argues in Willful Ignorance (paraphrasing for brevity; read and come to your own conclusions) we’re using statistical and probabilistic tools in economics that were purposely left crude by the mathematicians that defined them, as they were defined to handle abstract quantities.

   Applying them to human economics means we’re mathematically accepting rounding errors that leave people behind.

   That is the pattern we have to break politically. We spend a lot of time emotionally promoting certain stats and hand waving away at a number like “2% reduction in poverty” still means we’re leaving millions behind.

   Not having services like universal healthcare is clearly politically gamed, but we refuse to see it as anything like a government bombing civilians.

   Show morethread