Machine Learning Street Talk
4 HN recommenders · recommended in 6 months, 2021–2024
2:46:27 · 154K views · Jan 20, 2021
A long Machine Learning Street Talk interview with Kenneth Stanley, creator of NEAT and novelty search, about his book on why greatness cannot be planned. They discuss open-endedness, the tyranny of objectives, serendipity and the evolution of ideas.
Recommended to people who find it inspiring or perspective-changing on pursuing goals, open-ended search and AI beyond optimization.
9 comments from Hacker News
> The signal might be good but is it the signal you should be focusing on? There is an amazing amount of interesting stuff being discovered all the time. But is it helping you achieve concrete goals? My point is that putting everything through the filter of "does this directly help me achieve my goals?" isn't the best way to pursue your goals, assuming we're talking about big goals where the path to achieving them isn't clear. I recommend checking out this podcast episode called "Why Greatness Cannot be Planned". It really changed my perspective on achieving my long term goals: this video Also on Twitter, yeah some people don't only tweet things in their domain. But I've found that some people do reliably stick to a single domain. Though I don't really follow science people on Twitter.
I think it's mainly a competitive & cooperative self replicator. The brain exists because previous generations did well. Do we need to infuse "antenna magic" into our brains, aren't they marvelous enough? I found this video about open endedness (contrasting evolution vs optimization) in the brain and AI very inspiring: this video
Lehman & Stanley concerning how explicit objectives encourage effort on exploit rather than explore. AI is progressing using objective functions that can be expressed in a concise form (energy minimization, gradient descent, etc.). A similar mechanism has been at work in industry since Ford and Winslow, from time-and-motion to jit-supply-chain, all under the short-term profit incentive. These are narrow known objectives with a desire to get relatively quick results. But perhaps the big leaps are not achieved by climbing local hills. The observation harks back to Thomas Kuhn, who described the activities of normal science and great leaps - scientific recolutions. Obviously there are further philosophical implications, but also economic and administrative cliffs - financialization of the economy, bloat in university admin, group-think in academic tracks, (US) healthcare administration - all lead to temporary money extraction, with cautious risk-averse investments in bits. But there is not so much free-riding backing of mavericks, long-shot bets on contrarian ideas, and speculative gambling on profound, but expensive and difficult, inventions from atoms (see Peter Thiel - "courage is in far shorter supply than genius" ). Why Greatness Cannot Be Planned is Stanley's latest book (2015). Here are some excellent long-form podcast interviews with Stanley on MLST: Is getting a pay rise a step to becoming a billionaire? #38 Prof. KENNETH STANLEY - Why Greatness Cannot Be Planned (2021)
> Anyways, my point is, none of this required math or genius or particularly demanding thought. It was basically let's tinker with this until we find a way that's better, using guess and check. No equations needed. I get that you'd like to have a clear theoretical basis for what works and we're far from there. But in the meantime we're stumbling in the dark, discovering tricks and forming intuitions, not knowing even where the road is going to lead us. This is an evolutionary process of ideas, similar to biological evolution that managed to make us. If you know where you're going you can optimise your actions but when you don't even know what might be useful later on, then all attempts are good. They increase diversity and discover blind spots. Some of them will be the stepping stones for the future, but we can't say in advance which and how. Link to a long discussion about the evolution of ML ideas and the book "Why greatness cannot be planned" by Kenneth Stanley - this video
You're right, the search space is deceptive and going directly towards the goal might lead to a dead end. Only in hindsight we can tell which stepping stones were essential. Immediate results are the best, but in the meantime we need to focus on novelty and interestingness. A great video about open-endedness (warning - very long) with Kenneth Stanley, author of the book "Why Greatness Cannot be Planned" this video