Expert Political Judgment

What Hacker News says

1. RationalDinoNov 2023
   It is both an invitation and a conclusion.

   As you note, we need to decide who to trust, and we can't do sufficient research. So we need heuristics.

   The heuristics that I recommend are these:

   1. Certainty, especially if there is debate among experts, indicates likely cognitive bias.

   2. The ability to discuss how multiple perspectives would lead different people to different conclusions, suggests good reasoning.

   3. A documented history of quantitative reasoning, along with data showing historical accuracy, is the best evidence of a trustworthy source.

   I base those heuristics on the research in https://www.amazon.com/Expert-Political-Judgment-Good-Know/d.... Applying this heuristic to "sounds certain" results in a conclusion, that is an invitation to further research that may or may not happen.

   Show morethread
2. RationalDinoOct 2023
   This relates to one of the topics I want to get to on https://rationaldino.substack.com/ but which will take a while.

   There is a very simple litmus test to tell which are likely to fade, versus which aren't. How certain are they?

   https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... reports on a long-term study about pundits. Basically they can be divided into two groups:

   1. Hedgehogs. Have one overarching theory that they are certain of.

   2. Foxes. Can pick and choose from a variety of sources.

   Both groups are smart, informed, and interesting. But when you follow them for a long time, a clear pattern emerges.

   - Foxes do much better at making future predictions that come true.

   - Hedgehogs become far more popular pundits, and generally wind up getting paid far better. Most pundits with popular shows are hedgehogs.

   Why? My theory is that we listen to pundits because it is comfortable to outsource our thinking to them. We find that comfortable if they are smart, well-informed, and certain. It is easy for us to think, "Well if this smart and well-informed guy is so certain, I'd surely agree if I did the work. So now I don't have to bother."

   There is a problem here. We become certain when it is easy for us to think a thing true, and hard for us to think it might be false. We feel that the evidence is truly overwhelming. It may be overwhelming. But it is more likely that we're simply being intellectually dishonest. So we actively choose intellectually dishonest pundits who agree with our presumptions, and then become sure that they are right. We enjoy listening to them. But, being intellectually dishonest, they are probably wrong. And now we're emotionally committed to their brand of insanity!

   Try this rule of thumb out. Assume that a person who is certain, is probably wrong. And when you find yourself feeling certain, nurse that little doubt about how you REALLY know. It takes time, but consistently making this choice can change your life. For a start, you'll start actually thinking about things that you currently only think you're thinking about.

   Show morethread
3. mcshicksOct 2021
   Specifically in terms of being predictive you might be interested in reading Expert Political Judgment: How Good Is It? How Can We Know? by Philip E. Tetlock if you are not familiar with it already. He tried to empirically test experts political judgement and the main conclusion was pretty much everybody was bad. He now runs the good judgement project which trying to measure good forecasting skills to figure out who really is good at it. It was very eye opening to me at the time.

   https://press.princeton.edu/books/hardcover/9780691178288/ex...

   Show morethread
4. jacques_chesterJul 2019
   Yes, though speculative investments can last for a long time. There's also the confluence of macroeconomic factors: the emergence of China, continuing massive savings rates in Japan with little to invest in domestically, the decision by Saudi Arabia to open a massive sovereign wealth spigot and the quantitative easing that occurred after 2008. Financial capital is incredibly cheap by historical standards.

   That said, I find myself using one of the classic defences of a confident prognosticator: I am right / I will be right, except about the timing[0]. But to make more verifiable predictions I would need to start assigning probabilities to particular dates.

   [0] Tetlock's Expert Political Judgment: How Good Is It? How Can We Know? is still a brilliant and amusing empirical study of opinionated forecasting: https://www.amazon.com/Expert-Political-Judgment-Good-Know/d...

   Show morethread
5. mcshicksMay 2016
   Philip Tetlock has done a lot of research on objective measures of political expertise, specifically the ability to make objective predictions about future outcomes and likelihoods. He wrote a pretty famous book on it. The basic conclusion is nobody is really that good, but some people are objectively better than others. If you are really interested in this he has as website called "good judgement project" where you can test your ability to predict against other people on the site over time.

   http://www.amazon.com/Expert-Political-Judgment-Good-Know/dp...

   https://www.gjopen.com/

   Show morethread
6. adamSep 2015
   You should read Tetlock's other work about foxes and hedgehogs in Expert Political Judgment. It presents a theory of why some people are indeed better forecasters than others. It's not lucky or genius, it's more about people who are able to consider several different data points, consider historical precedent, and make a relatively unbiased judgment about what's going to happen. These are the same types of people he sought out for the Good Judgment project, and its these same types of people who were considered Superforecasters making hundreds of forecasts over the course of a year about geopolitical issues.
   Show morethread