The Flaw of Averages

What Hacker News says

1. kqrApr 2022
   Some authors take multiple books to fully flesh out the ideas outlined in the first, so there are some doubles recommended here.

   I also found it impossible to rank books, so instead I took roughly the top 20, grouped them into themes, and then ranked the themes based on how important I think they are.

   1. Deming is the primary writer of the type of management we need to be competitive: Deming, The New Economics & Out of the Crisis.

   2. There is a wide variety of ways for humans to organise. Nobody is forcing the default capitalist bureaucracy on you, so be creative: Graeber, Bullshit Jobs & Dawn of Everything & The Democracy Project.

   3. How people are motivated: Pink, Drive.

   4. How to create a high-trust, less authoritarian, open, collaborative environment within a traditional hierarchy: Marquet, Leadership is Language.

   5. Communication: Voss, Never Split the Difference. Faber, How to Talk So Kids Will Listen.

   6. How to help organisations who don't want to admit what their problem is: Weinberg, Secrets of Consulting.

   7. How to structure organisations and processes to deliver maximum value: Reinertsen, Principles of Product Development Flow. Ward, Lean Product and Process Development. Womack, Machine That Changed the World.

   8. How to innovate cheaply: Westrum, Sidewinder.

   9. How to estimate things properly: Hubbard, How to Measure Anything. Tetlock, Superforecasting. Savage, The Flaw of Averages.

   10. How to train people to be experts at things: Hoffman, Accelerated Learning.

   11. How people fail at decisions: Kahneman, Thinking, Fast and Slow & Noise.

   I had to leave out a few about statistics, causality inference, project portfolio investment that didn't make the top 20 cut.

   Show morethread
2. kqrAug 2021
   This kind of stuff is all I've been reading recently. I suggest the following, maybe possibly even in the order given.

   It's difficult to give a single recommendation, because it's a broad topic that branches off into several important directions. A continuously improving learning organisation takes many interlocking ideas to work, relating to long-term value powered culture, economic and probabilistic reasoning, uncertainty reduction techniques, including running experiments and gathering data on them, and analysing that data. But more important than running experiments are figuring out where the heck you stand and where you want to go. Just knowing that gets you halfway there.

   ----

   These first four I would consider mandatory for anyone interested in this kind of stuff.

   - Out of the Crisis or The New Economics (I keep confusing these with each other because I read them after each other -- well, you've probably read these already given your comment)

   - The Machine That Changed the World (this seminal study introduced the industry to the idea of lean manufacturing, and it tells you exactly how important the ideas of lean/flow are)

   - Principles of Product Development Flow (the author is a guru in applying lean/flow ideas to product development and teaches you to not blindly apply lean manufacturing ideas to development, but to adapt them to the economic conditions of development)

   - Lean Product and Process Development (an opinionated but very intriguing view on how product development should be done, with focus on building a learning organisation)

   ----

   These next ones are a little less mandatory and not necessarily extremely topical, but still interesting.

   - How To Measure Anything (before you collect data the most expensive way possible -- with experimentation -- try estimation using proper techniques first; it's really cheap and surprisingly good)

   - Understanding Variation (a good introduction to statistical process control, which helps you interpret your operational experience)

   - Learning To See (value stream mapping is a way to visualise some wastes in processes)

   - Accelerate (huge, impressive study of what high performing development organisations do differently)

   - Drive (primer on the psychology of motivation)

   - Trustworthy Online Controlled Experiments (there are probably better books on experimental design -- I've been eyeing some of Deming's myself -- but this one is probably as good a primer as any)

   - The Flaw of Averages (to get more accurate predictions, focus on distributions and ranges, not single numbers)

   - Toyota Kata (a rote way to build a culture of continuous improvement)

   - Making Software: What Really Works, and Why We Believe It (general high-quality exposition on evidence-backed techniques -- maybe not strictly in the scheduling/project management realm)

   - Rapid Development (like Making Software except older and thicker)

   - Turn the Ship Around (case study in applying some of these principles to completely change the performance of a team)

   - Lean Enterprise (more on how to adapt lean ideas to the whole development organisation, with case studies, IIRC)

   - Willful Ignorance (what are probabilistic statements, really?)

   ----

   I have many more recommendations if you want to go deeper into a specific sub-topic, but this is probably enough for a cursory comment.

   Show morethread
3. refrigeratorApr 2020
   Hey folks, I'm one of the authors of the post. Would love to hear people's thoughts, and happy to answer any questions :)

   If you liked this, then I'd definitely recommend reading "The Flaw of Averages" by Sam Savage. It talks about this stuff in a lot more detail, and the style is very entertaining — not at all what you'd expect from a statistics book.

   thread
4. CodeCubeSep 2012
   Wow, great discussion on risk. The only thing I would add to the discussion is that before you try to model risk, you should read 'The Flaw of Averages', by Sam Savage - http://flawofaverages.com/

   Put simply, any real risk model should probably avoid being a 'single number', but a range of probabilities. This models the distribution of possibilities much better and helps you make better decisions.

   thread
5. simulateAug 2012
   Stanford's Sam Savage has written an excellent book on the topic of disaggregating average data and mistakes made from using averages. Here's a summary of the book on his website: http://www.stanford.edu/~savage/flaw/

   and here's a link to his book, titled The Flaw of Averages: http://www.amazon.com/Flaw-Averages-Underestimate-Risk-Uncer...

   thread
6. jlduggerSep 2026
   About two thirds of the way through "Flaw of Averages". Thankfully the chapters are short but it's written for business students and is aggressively anti-jargon, which can be annoying at times.

   It started off promising, with some hints about monte carlo methods implemented in spreadsheets, but kind of detours into statistics basics, layman summaries of the author's prior scholarly work, and various business consulting anecdotes. I did not, for example, expect to read a chapter on the FASB-II's flaws regarding options pricing. And a few are downright concerning a decade later. I would not, for example, brag about advising Wells Fargo executives on their employee incentive programs after the cross-selling scandal came to light.

   The final chapters, which I have not yet gotten to, supposedly cover the solution to the flaw of averages. Given the book is 15 years old, I'm expecting it to be pretty dated implementation wise, but I might be able to apply it to Prometheus histograms or t-digests. And I've learned a few things, like Jensen's inequality, and found a few sheets demoing a sampling approach.

   Show morethread