Insider Tech
4 HN mentions · mentioned in 4 months, 2017–2022
1:51 · 11M views · Jul 12, 2017
A two-minute Insider Tech clip on DeepMind's simulated agents that learned to walk, run, jump and climb without guidance, with the goofy resulting gaits.
HN uses it as a quick example of reinforcement learning agents acquiring locomotion in simulation, rather than recommending it as a deep resource.
4 comments from Hacker News
If you watch videos of AIs learning to run [1] or playing Mario [2], you see that there's definitely "try random shit until your reward function gives you positive feedback" as a method for training AIs. The difficult part is developing the reward function. There's a lot of intelligence in our hormone-based incentive system - that we feel pain, that we feel sad, etc. Many of those emotions are pre-programmed. If we find a way to design reinforcement systems with the right goals, we can do it like mother nature did it for several billion years. But it's still a ton of computational power required. If we look at nature as a massively parallelized computational system that defined "just live" as the reward (not because it "wants" it, but because it just emerged), it shows us how much power we would need if we would try to build a completely random process that gains intelligence. I think your proposed method works ("just assemble random solutions and test them") because we've already seen it in nature. But it takes a lot of time and energy, regardless of our computational power. I think we're faster when we find the right boundaries and reward functions instead of randomly trying stuff. [1]: this video [2]: [LINK]
It didn't annoy me, but it definitely gave off a Disney Trailer vibe. I can just hear the deep announcer voice over the video coming in: "Coming Winter 2017, A movie about adventure, friendship, AND simulated self-play! Join Burp, Jenny and Bop on a fantastic adventure .... blah blah ..." etc.. hah Heres an example of good, unobtrusive but fun music for showing AI bots: this video
Hardware is hard. Making an AI that learns to control a purely virtual body appears to be a fairly standard project. 1) [LINK] 2) [LINK] 3) [LINK] 4) this video
I'm not sure about doing that with physical robots, but I've seen a couple interesting iterations using simulated robots this video