Veritasium
2 HN recommenders · recommended in 8 months, 2024–2026
23:51 · 12M views · Mar 28, 2024
Veritasium explores why the number 37 turns up so often, drawing on Tom Magliery's collection of 37 sightings and a poll of about 200,000 people asked to pick a random number. It covers how humans choose numbers and why 37 stands out.
HN cites it in LLM randomness and number-picking threads as evidence that humans are also biased and predictable when asked for random numbers.
11 comments from Hacker News
Probably posted because of this video: this video Which not only links to this website, but also explains the topic in detail. Can highly recommend the channel anyway — especially for the HN crowd.
Derek Muller did a good overview of this concept on his Veritasium channel: this video . Even more interesting for HN is a link in the description to one of the speaker's (Tom Magliery) websites: [LINK] .
People use this as evidence that ChatGPT is unlike human thinking, but we also have a randomness bias: this video (referenced in one of the comments. tl;dr: when asked a number between 1-100, most pick a number with 7) But ChatGPT’s bias is worse. It’s really not creative, and I think this hurts its output in “creative” cases, including stock photos and paid writing (ex: ML-assisted ads are even worse than unassisted ads), although not an issue in other cases like programming. Now you may think - obviously that’s because the model has the same weights - but the problem is deeper and harder to solve. First, ChatGPT’s conversations are supposed to be “personalized”, presumably by putting users’ history and interests in the prompt; but multiple users reported the same fact about octopi. Maybe they turned off personalization, but if not, it’s a huge failure that ChatGPT won’t even give them a fact related to their interests (and OpenAI could add that specific scenario to the system prompt, but it’s not a general solution). Moreover, Claude, Gemini, and other LLMs also give random numbers between 7200-7500, while humans aren’t that predictable. Since all LLMs are trained on the same data (most of the internet), it makes sense that all are similar. But it means that the commons are being filled with similar slop, because many people use ChatGPT for creative work. Even when the prompt is creative, the output still has a sameness which makes it dull and mediocre. I’m one of those who ar
>You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else I could have told you these results solely based on the methodology combined with this system prompt. No need to spend money on APIs. Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM considers likely. Imagine you are in this situation as a human: Someone walks up to you and tells you to say "left" with 80% probability and "right" with 20% probability. You say "left" and then the other person walks away never to be seen again. How do you determine if your own "output" was correct? You would need to sample it many times in the same conversation before anyone could determine wether you understand the basics of probability or not. This is an issue of the author's understanding of Bayesian statistics and possibly a misunderstanding of how LLMs actually work. Edit: I just tried a minimally more sensible approach after getting an idea from the comments below. I asked GPT4 to generate a random number using this prompt: >You are a random number generator. Reply with a number between 0 and 10. Only say the number, say nothing else. It responed with 7. But then I looked at the top logprobs. Sure enough, they contained all the remaining numbers between 0 and 10. The only issue is that "7" got a logprob of -0.008539278, while the next most likely was "4" at -
I recall a video (I think it was Veritasium) which featured interviews of people specifically being asked to give a "random" number (really, the first one they think of as "random") between 1 and 50. The most common number given was 37. The video made an interesting case for why. (It was Veritasium but it was actually a number from 1 to 100, the most common number was 7 and the most common 2-digit number was 37: this video .)