
What gets people are incorrect procedures. To get a sense of all the ways in which an experiment can go wrong, I'd recommend reading more traditional texts on experimental design, survey research, etc.
- Donald Wheeler's Understanding Variation should be mandatory reading for almost everyone working professionally.
- Deming's Some Theory of Sampling is really good and covers more ground than the title lets on.
- Deming's Sample Design in Business Research I remember being formative for me also, although it was a while since I read it.
- Efron and Tibshirani's Introduction to the Bootstrap gives an intuitive sense of some experimental errors from a different perspective.
I know there's one book covering survey design I really liked but I forget which one it was. Sorry!
I think perhaps the best bang for your buck could be Wheeler's Understanding Variation -- but that is based mainly on vague memory and skimming the table of contents. I plan on writing a proper review of that book in the coming year to make certain it is what I remember it to be.
I think the earlier works by Taleb also touch on this (Fooled by Randomness seems to have it in the title).
But then I strongly recommend branching out to places where these fundamentals are used, to cement them:
- Anything popular by Deming (e.g. The New Economics)
- Anything less popular by Deming (e.g. Some Theory of Sampling)
- Moneyball
- Theory of Probability (de Finetti)
- Causality (Pearl)
- Applied Survival Analysis
- Analysis of Extremal Events
- Regression Modeling with Actuarial and Financial Applications
The more theoretical and applied books are less casual reads, obviously. They also happen to be the directions in which I have gone -- you may have more luck picking your own direction for where to apply and practice your intuition.
Edit: Oh and while I don't have a specific book recommendation because none of the ones I read I have good opinions on, something on combinatorics helps with getting a grasp on the general smell of entropy and simpler problems like Monty Hall.