David Chapman writes about meta-rationality, the limits of probability and formal logic, and how rationality works in practice, including a book-in-progress.
HN commenters recommend it as clarifying, especially for scientists and engineers, and point to its pieces on Bayesianism and maps versus territory.
Sorry, didn't see your reply earlier. Don't have much time to explain, but for context, you probably want to read "a first lesson in meta-rationality" [0]. It doesn't talk much about categorization directly, but it broadly discusses the limits of rationalism, and starts to explore the "solutions" / alternative perspectives (that the author decided to label as "meta-rationalism"). The categorization problem is one of the points where you can see the "limits" of rationalism. It's never discussed to much or too explicitly in these terms, though. I advice you to read the whole thing attentively, and when you are done, go back to review the section on nebulosity, and specially this quote:
>> One has to be able to “bend” concepts, when it is appropriate. Nothing should be absolutely rigid. On the other hand, things shouldn’t be so wishy-washy that nothing has any meaning at all, either. The trick is knowing when and how to slip one concept into another.
And reflect again at this point about what you have read:
>> The point of these examples is that what counts as an “object” depends on the context.
...while thinking explicitly about the categorization problem. In computer science and AI, in particular, this problem might appear even more clearly, specially in symbolic AI. If you try to create a symbolic AI system, or if you are an experienced programmer and have a solid grasp of the limitations of abstraction, it all floats around this main issue of nebulosity / categorization…
“There is always uncertainty in science, but real science gradually establishes some stable facts; it eventually strongly supports some theories and conclusively dismisses others. … Nutrition has made no progress. It has discovered no stable facts. Everything nutritionists have said, they have said the opposite ten or twenty years later (if not much sooner). They literally know nothing. After a century of countless experiments, the most common, most basic problem they’ve addressed—the optimal ratio of fat, protein, and carbohydrate—is completely unsolved. If they can’t figure that out, anything more sophisticated seems hopeless. … My point is not that nutrition is bad science. Unquestionably, it is bad science; a competent statistician, looking at the design of most experiments, will immediately say “this is meaningless; you can’t learn anything this way.” It’s worse than just incompetence … even the best studies have been useless. There seems to be something fundamentally wrong, such that doing the same sort of science better wouldn’t help.”
https://metarationality.com/nutrition
This is a deep topic worthy of an essay, the kind of essay pg might write! The concise reply is, when you catch yourself in dichotomous thinking, you should assume you have misunderstood or oversimplified.
Sometimes you have to go with your misunderstanding or oversimplification to make a decision or to make progress, but keep in mind you are doing so based on beliefs which are unlikely to correspond to reality.
It's striking that so many mental health difficulties are characterized by dichotomous thinking. [1]
Rationality itself can be profitably critiqued at the meta-level.[2] So, don't be an asshole, unless you need to be. Do you see the world as consisting of assholes and non-assholes? How does that feel? What are the advantages of that view? What are the disadvantages? Is considering that question worth your time?
[1] https://www.psychologytoday.com/us/blog/she-comes-long-way-baby/201506/what-is-wrong-dichotomous-thinking
[2] https://metarationality.com/introduction
EDIT: Fixed link, spacing
But how do you know that "informal logic" has anything to do with formal logic? Outside of math, formal logic seems actively misleading. For example, the "principle of explosion" (from a contradiction follows anything) is definitely not how people think.
Philosophically, I prefer the reasonable / rational / meta-rational framework explained by David Chapman. [1] But if you want to know how people ordinarily think, better to study how they think.
(And in practical everyday use on the Internet, I find logic mostly useful for for shooting down attempts at inappropriately applying logic to conclude things that you can't safely conclude.)
[1] https://metarationality.com/reasonableness-aspects
Yet another nutrition study.
here's a counterpoint:
> Nutrition has made no progress. It has discovered no stable facts. Everything nutritionists have said, they have said the opposite ten or twenty years later (if not much sooner). They literally know nothing. After a century of countless experiments, the most common, most basic problem they’ve addressed—the optimal ratio of fat, protein, and carbohydrate—is completely unsolved. If they can’t figure that out, anything more sophisticated seems hopeless.
https://metarationality.com/nutrition
While I fully agree withtthe gist of this quote, I think one major point is wrong: the idea of an optimal ratio between the macronutrient is probably bullshit in itself. In fact, the idea of categorizing food into such simple categories (three macronutrients + vitamins and minerals) at all seems wrong. Fiber plays a major role, different fats and different carbs have widely different effects, probably different proteins too.
Note: the source of the quote for anyone curious to read more:
https://metarationality.com/nutrition