
Oooh, thats a good one. Next read the Architects paradox, Why Greatness cannot be planned and Understanding Variation and your views of the world will be forever altered. Or pick up "Architecture Modernization" by Nick Tune if you want more tools to do stuff and if you do not want to achieve enligntenment.
_____
Where did you acquire cheap ancient coins? ebay? May be cool to get some for my dnd group
My two questions (a) and (b) were not rhetorical. Let’s get concrete.
a) You are advising a company to “check back after a certain period”. After the certain period, they come back to you with the following graph:
https://commoncog.com/content/images/2024/01/prospect_calls_...
“How did we do? Did we improve?”
How do you answer? Notice that this is a problem regardless of whether you are a big company or a small company.
b) 3 months later, your client comes back and asks: “we are having trouble with customer support. How do we know that it’s not related to this change we made?” With your superior experience working with hundreds of startups, you are able to tell them if it is or isn’t after some investigation. Your client asks you: “how can we do that for ourselves without calling on you every time we see something weird?”
How do you answer?
(My answers are in the WBR essay and the essay that comes immediately before that, natch)
It is a common excuse to wave away these ideas with “oh, these are big company solutions, not applicable to small businesses.” But a) I have applied these ideas to my own small business and doubled revenue; also b) in 1992 Donald Wheeler applied these methods to a small Japanese night club and then wrote a whole book about the results: https://www.amazon.sg/Spc-Esquire-Club-Donald-Wheeler/dp/094...
Wheeler wanted to prove, (and I wanted to verify), that ‘tools to understand how your business ACTUALLY works’ are uniformly applicable regardless of company size.
If anyone reading this is interested in being able to answer confidently to both questions, I recommend reading my essays to start with (there’s enough in front of the paywall to be useful) and then jump straight to Wheeler. I recommend Understanding Variation, which was originally developed as a 1993 presentation to managers at DuPont (which means it is light on statistics).
I would not want to go through life not having read those -- much less run an early-stage company!
What gets people are incorrect procedures. To get a sense of all the ways in which an experiment can go wrong, I'd recommend reading more traditional texts on experimental design, survey research, etc.
- Donald Wheeler's Understanding Variation should be mandatory reading for almost everyone working professionally.
- Deming's Some Theory of Sampling is really good and covers more ground than the title lets on.
- Deming's Sample Design in Business Research I remember being formative for me also, although it was a while since I read it.
- Efron and Tibshirani's Introduction to the Bootstrap gives an intuitive sense of some experimental errors from a different perspective.
I know there's one book covering survey design I really liked but I forget which one it was. Sorry!
I think perhaps the best bang for your buck could be Wheeler's Understanding Variation -- but that is based mainly on vague memory and skimming the table of contents. I plan on writing a proper review of that book in the coming year to make certain it is what I remember it to be.
I think the earlier works by Taleb also touch on this (Fooled by Randomness seems to have it in the title).
But then I strongly recommend branching out to places where these fundamentals are used, to cement them:
- Anything popular by Deming (e.g. The New Economics)
- Anything less popular by Deming (e.g. Some Theory of Sampling)
- Moneyball
- Theory of Probability (de Finetti)
- Causality (Pearl)
- Applied Survival Analysis
- Analysis of Extremal Events
- Regression Modeling with Actuarial and Financial Applications
The more theoretical and applied books are less casual reads, obviously. They also happen to be the directions in which I have gone -- you may have more luck picking your own direction for where to apply and practice your intuition.
Edit: Oh and while I don't have a specific book recommendation because none of the ones I read I have good opinions on, something on combinatorics helps with getting a grasp on the general smell of entropy and simpler problems like Monty Hall.
I've long wanted to write an open article series for someone like you but never gotten around to it. There's so much information out there, but you sort of have to piece it together on your own, which is suboptimal.