Veritasium
3 HN recommenders · recommended in 5 months, 2015–2023
4:44 · 17M views · Feb 24, 2014
A Veritasium video that runs the 2-4-8 rule-guessing puzzle on people to show how they test only examples that confirm their hypothesis. It uses this to illustrate confirmation bias and falsification.
Recommended as a clear demonstration of confirmation bias and the value of trying to break your own hypothesis.
8 comments from Hacker News
Experimental science is all based around ruling out the null hypothesis. For that it needs to be falsifiable . The hypothesis 'the dog can differentiate between A and B' is very hard to falsify. Because the dog could differentiate, but choose to not act. You would need a fairly complete understanding of the dogs mental workings, and scanners to study them. On the other hand, 'the dog can't differentiate between A and B' is much easier to falsify. If you repeat an experiment a sufficient amount of times and the dog consistently has different behavior between A and B, you can rule this null hypothesis out. That is the real reason we always 'assume the worst'. Because 'assume the worst' is the easiest to scientifically rule out. EDIT: I would suggest this video by Veritasium, which also touches on this, at a very fundamental level: this video .
Veritasium has a great video[1] about people’s cognitive bias towards using examples that prove their mental model, rather than using examples that disprove their mental model. But if you want to actually confirm you have accurate understanding of something, testing examples that don't fit your metal model is the workable methodology (i.e. use a null hypothesis). Testing examples that do match your mental model only proves your model partially matches the actual model, but it does very little to actually identify misunderstandings, or improve your understanding. [1] this video
Veritasium, a pretty interesting youtube channel, posted a video on this experiment a while back. I found the discussion afterwards to be more thought-provoking than this article. this video
Derek Muller of Veritasium has done this test over a year ago on people in one of his videos: this video I've given the test to various people since then and never once came across someone who'd guess it right away, or within a short period of time. The breaking time came usually after a few minutes when they gave up and started throwing out random numbers that coincidentally did not meet the rule. Once you hit the first "No", it took a very short time to figure out the rule for almost everyone.
This is now the fourth time I've seen this exact rule. Another example: this video It's probably not "fair" to say I got it in zero... but I did. :) Now that the NYT has done it, this puzzle has probably attained enough popularity now that you really ought to change it up a bit now if you're going to run it yourself. Granted, the space of hypotheses as simple as "increasing/decreasing" is pretty small, but your ability to fool people with the first sample run is almost unbounded, so that helps.