Dwarkesh Patel runs a podcast and blog with long interviews and essays on AI, history, economics and learning.
Commenters recommend his episodes, especially those with Sarah Paine on history and Andy Matuschak on self-learning, and cite his Zuckerberg interview.
> [T]he mythos of people having popped out from the same area or region that they their most recent ancestors have lived […]
>> Genetic studies have put an end to that kind of speculation. Only crackpots and religious nuts are supporting alternatives.
David Reich (author of "Who We Are and How We Got Here: Ancient DNA and the new science of the human past") says[^0]:
" The modern human lineage, leading to the great majority of the ancestors of people today, was probably in sub-Saharan Africa for the last 500,000 years at least. It might be much more. Certainly our main lineage was in Africa, probably 3-7 million years ago.
But in a period between about 2 million to 500,000 years ago, it's not at all clear where the main ancestors leading to modern humans were. There were humans throughout many parts of Eurasia and Africa with a parallel increase in brain size and not obviously closer ancestrality to modern humans in one place than in the other. It's not clear where the main lineages were. Maybe they were in both places and mixed to form the lineages that gave rise to people today.
There's been an assumption where Africa's been at the center of everything for many millions of years. Certainly it's been absolutely central at many periods in human history. But in this key period when a lot of important changes happen—when modern humans develop from Homo habilis and Homo erectus all the way to Homo heidelbergensis and the shared ancestor of Neanderthals, modern humans, and…
I hope you'd at least concede without needing bulletproof evidence that it's a more reasonable possibility than the second coming of Jesus.
The best evidence I can offer is the continued saturation of benchmarks, the continued effectiveness of scaling laws with respect to improvements in general reasoning, the massive improvements in algorithmic capabilities which lead to better use of existing models and inference-time compute, and the overall lack of fundamental blockers articulated that differentiate AI and human reasoning that can't be fixed via these developments.
Every LLM of a given size is invariably at or near the state of the art with respect to all benchmarks available to test general reasoning. GPT-4 was the first model in the >1T parameter range, and every model that emerged around that level (Claude 3 Opus, Gemini Ultra) reached that level of performance in MMLU, MATH, etc. All subsequent improvements in models have been distillations and algorithmic improvements, synthetic data, etc. but so far a ~10T model has not been attempted, though it is likely around the scale for the upcoming GPT-5, Claude 4 and Gemini 2 models.
Sources: https://www.dwarkeshpatel.com/p/will-scaling-work https://www.lesswrong.com/posts/arPCiWYdqNCaE7AQv/superintelligent-ai-is-possible-in-the-2020s https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/61d936fba2567df92a62dc70fe8d1fc9b381078b91445fd50d8dea228ced3685/nzajcospcpkcert7cuup
Don't entirely disagree but I think "mad" is an unfair characterization and simplifies how the fascinating competitive dynamics have played out that resulted in the Llama models.
Meta as well as Amazon and MS have been strategically constrained by not owning the mobile platform and they've been pretty consistently clear about that. https://x.com/ianand/status/1753425306181116394
Like it or not, Apple's ATT move (i.e. "Ask App Not to Track") hit their revenue in real dollars and stock price significantly. Not only that, the ensuing chaos helped pave an opening for the rise of competitor TikTok:
> Meta, formerly known as Facebook, said that one setting alone cost the company an estimated $10 billion. Its stock value has plunged 70% this year. But ATT had another side effect, one that got far less attention than Meta’s troubles. Apple’s iPhone privacy setting gave TikTok a significant leg up in its fight for social media dominance. (from https://gizmodo.com/how-apple-s-ask-app-not-to-track-prompt-helped-tiktok-1849772915)
Their savior to improve monetization in the face of ATT turned out to be AI:
> Further integration of AI helped drive Meta’s first revenue increase in three quarters, the company said on Wednesday. Reels monetization is up over 30% on Instagram and over 40% on Facebook on a quarterly basis as AI plays a larger role in the platforms. (from…
>On the plus side, some software engineers save a couple minutes here and there with a slightly nicer autocomplete, white collar workers can get some advice on email style, and maybe it helps with sifting through millions of pages of legal discovery.
Lol get a grip. Jeff Dean just said 25% of the code being checked into Google's internal codebase is now AI generated. If you think it's just kids using it for homework and engineers saving a few minutes here and there you're kidding yourself.
https://www.dwarkeshpatel.com/p/jeff-dean-and-noam-shazeer
>I am hopeful for some of the more advanced applications, e.g. proof assistance in math,
"I expect, say, 2026-level AI, when used properly, will be a trustworthy co-author in mathematical research, and in many other fields as well." You just aren't in the same universe as what the top experts are saying about this tech.
https://unlocked.microsoft.com/ai-anthology/terence-tao/
You seem like one of those people that used GPT3.5 once and hasn't updated at all since then. You're criticizing the wright flyer and we're already at turboprops.
Self-imposed limitations are key to being effective. I was referring to pointless self imposed limitations.
I was reading an interview with Andy Matuschak[1].
> One of the things that I think is kind of weird about this memory system stuff, or like memory champions, or something like that is “Oh, if you do these things, will you start to forget other normal human stuff?” And what's weird is, no. I've been doing this memory system stuff for years and I just know more stuff now. This is aligned with the experimental literature, which seems to suggest that, there's probably upper bounds but we're not close to them. Some of these memory champions have memorized maybe two orders of magnitude more things than I have practiced. Certainly people who are multi-lingual have really, really absurd numbers of things memorized. So there isn't a resource management argument.
The notion that stuffing your brain with trivia will be damaging is merely an unwarranted fear.
[1] https://www.dwarkeshpatel.com/p/andy-matuschak
As far as I (ex-ML researcher) know, the main technological case that LLM performance will hit a limit is due to the amount of text data available to train on is limited. The ways these scaling laws work is they require 10x or 100x quantity of data to see major improvements.
This isn't necessarily going to limit it though. It's possible there are clever approaches to leverage much more data. This could either be through AI-generated data, other modalities (e.g. video) or another approach altogether.
This is quite a good accessible post on both sides of this discussion: https://www.dwarkeshpatel.com/p/will-scaling-work