The cult of statistical significance

What Hacker News says

1. misiti3780Apr 2015
   I just finished a pretty interesting book on this topic:

   "The Cult Of Statistical Significance":

   http://www.amazon.com/Cult-Statistical-Significance-Economic...

   It basically goes through a bunch of examples, mostly in Economics, but in medicine also (Vioxx) where statistical significance has failed us and people have died for it. As someone who works with statistics for a living, I found to book interesting - but it was pretty depressing to find out that most scientist are using t-test and p-values because it seems to be the status quo and it is the easiest way to get published. The authors of this book suggest a few different things -- publishing the size of your coefficients and using a loss function. In the end, they make the point that statistical significance is different than economic significance, political significance, etc.

   Show morethread
2. amateurpolymathApr 2015
   There has been plenty of debate about this in other fields as well. Deirdre McCloskey and Stephen Ziliak have a particularly well-written paper titled "The Cult of Statistical Significance" on this very topic. Their main point is that statistical significance is meaningless without a discussion of magnitudes.

   [1] http://www.deirdremccloskey.com/docs/jsm.pdf

   thread
3. RA_FisherFeb 2015
   I have a simple criterion for a summary judgement of the reliability of results:

   a) Is the data made available? b) Is it a Bayesian analysis? c) Has a power study been offered?

   As a statistician, I have a keen awareness of the ways that p-values can depart from truth. You can see Optimizely's effort to cope (https://www.optimizely.com/statistics). You can read about it in The Cult of Statistical Significance (http://www.amazon.com/The-Cult-Statistical-Significance-Econ...). This Economist video captures it solidly (http://www.economist.com/blogs/graphicdetail/2013/10/daily-c...).

   The key component missing is a bias towards positive results. Most scientists only have two statistics classes. In these classes they learn a number of statistical tests, but much less how things can go wrong. Classic, "just enough to be dangerous."

   In order to cope, I have a personal set of criteria to make a quick first sort of papers. It's a personal heuristic for quality. I assume some degree of belief (Bayes, FTW!) that those that offer the full data set along side conclusions feel confident in their own analysis. Also, if they're using Bayesian methods, that they've had more than two stats classes. Finally, if they do choose Frequentist methods, a power study tells me that they understand the important finite nature of data in the context of asymptotic models / assumptions.

   I'd suspect that other statisticians feel this way, because I've heard that privately --- what do you think of my criteria?

   Show morethread
4. RA_FisherJul 2014
   The field receives a pretty scathing review in The Cult of Statistical Significance. The summaries offered there are pretty damning: http://www.amazon.com/The-Cult-Statistical-Significance-Econ...
   thread
5. RA_FisherNov 2013
   Yes, if you read further on the blog, you'll see I'm more than slightly denigrating. I read a book recently called The Cult of Statistical Significance and it's really turned me against Fisher. Not only do I see his methods as sub-standard to Bayesian analysis, he was really mean and disparaging himself to those folks. It's interesting, my username is what it is because he used to be my hero! Just 6 months ago, I really considered him to be the father of modern science. Now I basically see his work as mostly enabling mediocre scientists to rise through the ranks. This is what I mean by that: http://www.economist.com/blogs/graphicdetail/2013/10/daily-c...
   Show morethread
6. RA_FisherOct 2013
   I would highly recommend that anyone with further interest check out the book, "The Cult of Statistical Significance" It's eye opening.
   thread