Shane Legg (vetta.org)

What HN people wrote when they linked it

airgapstopgapMay 2023
Researchers aren't a monolith. Just because Gary Marcus (a complete fraud by the way, look up his "XProp" magitech he sold to Uber) pooh-poohed connectionist AI until ChatGPT came out doesn't mean nobody predicted gains from scaling; certainly many people deep in the know expected things to click. Certainly they expected that the basic trick of associative learning will be cracked. E.g. here's Shane Legg, December 2009 [1]:
> Conclusion: computer power is unlikely to be the issue anymore in terms of AGI being possible. The main question is whether we can find the right algorithms.
> One of the big things influencing me this year has been learning about how much we understand about how the brain works, in particular, how much we know that should be of interest to AGI designers. I won’t get into it all here, but suffice to say that just a brief outline of all this information would be a 20 page journal paper (there is currently a suggestion that I write such a paper next year with some Gatsby Unit neuroscientists, but for the time being I’ve got too many other things to attend to). At a high level what we are seeing in the brain is a fairly sensible looking AGI design. You’ve got hierarchical temporal abstraction formed for perception and action combined with more precise timing motor control, with an underlying system for reinforcement learning. The reinforcement learning system is essentially a type of temporal difference learning though unfortunately at the moment there…
nlJan 2014
It's difficult not to get caught up in the excitement around Deep Learning. The unsupervised nature of it at its best really does seem to approach human levels of learning (if not intelligence).
Also, I thought this was hilarious:
In a 2011 interview that predated DeepMind, co-founder Shane Legg said he gave only a 50 percent chance that human level-machine intelligence would exist by 2028.
ONLY?!? By 2028? That is a pretty radical prediction. Indeed, Legg says there is a 10% chance of human level intelligence by 2018, and 90% by 2050[1].
Legg has multiple papers published in the field of measuring machine intelligence, so I guess he has a pretty good view of the field.
If that 2028 prediction is even close to correct then I think setting up an ethics board now probably is the correct thing to do. (the notion that DeepMind asked that Google create an internal ethics board as a condition of the acquisition, as reported by The Information, had some AI researchers griping)
Edit: Legg's posting on his blog in 2011 deserves to be read:
I’ve decided to once again leave my prediction for when human level AGI will arrive unchanged. That is, I give it a log-normal distribution with a mean of 2028 and a mode of 2025, under the assumption that nothing crazy happens like a nuclear war. I’d also like to add to this prediction that I expect to see an impressive proto-AGI within the next 8 years. By this I mean a system with basic vision, basic sound processing, basic movement…
cute_boiJul 2026
No need stupid moving texts.
CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/
The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/
Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/
ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385
Multi-Scale Context Aggregation by Dilated Convolutions - https://arxiv.org/abs/1511.07122
Identity Mappings in Deep Residual Networks - https://arxiv.org/abs/1603.05027
Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595
Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391
Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473
Pointer Networks - https://arxiv.org/abs/1506.03134
Attention Is All You Need - https://arxiv.org/abs/1706.03762
The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/
Neural Turing Machines - https://arxiv.org/abs/1410.5401
A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427
Relational Recurrent Neural Networks -…
cousin_itJun 2011
> If easy instances are enough, what's the relevance of the provably ideal uncomputable problem, besides rhetoric?
It shows that there's a path to AI that's gated on solving formal math problems instead of imitating wishy-washy emotional humans. This point may be obvious to you, but apparently it wasn't obvious to Stross.
> If humans used easy instances, and they're the only example we know of that achieved I, how is that not weak evidence that AI through easy instances will pass through human-like I?
Humans are also the only animals that can play chess, but chessplaying software didn't pass through a human-like stage where they got nervous, forgetful, etc. My intuition says math can't be that radically different from chess (both involve search on trees with a complicated evaluation function), and we're only a handful of new insights from having computers decisively beat humans at math. And at that moment, by implication from my previous point, they might just also beat us at everything else :-) Remember that, on an absolute scale, humans are about as bad at math as you can be and still build a civilization (otherwise civilization would've happened at a lower level of development) and also that we probably suck at math because math is a conscious task (see Moravec's paradox).
Related link that should make you scared if you buy my reasoning: http://www.vetta.org/2009/09/monte-carlo-aixi/ .
gooobJul 2026
https://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
https://papers.nips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
https://arxiv.org/pdf/1512.03385
https://arxiv.org/pdf/1511.07122
https://arxiv.org/pdf/1603.05027
https://arxiv.org/pdf/1409.2329
https://arxiv.org/pdf/1512.02595
https://arxiv.org/pdf/1409.0473
https://arxiv.org/pdf/1506.03134
https://arxiv.org/pdf/1706.03762
https://nlp.seas.harvard.edu/annotated-transformer/
https://arxiv.org/pdf/1410.5401
https://arxiv.org/pdf/1706.01427
https://arxiv.org/pdf/1806.01822
https://arxiv.org/pdf/1704.01212
https://arxiv.org/pdf/2001.08361
https://arxiv.org/pdf/1811.06965
https://www.cs.toronto.edu/~hinton/absps/colt93.pdf
https://arxiv.org/pdf/math/0406077
https://scottaaronson.blog/?p=762
https://arxiv.org/pdf/1405.6903
https://onlinelibrary.wiley.com/doi/10.1002/047174882X.ch14
https://github.com/Bladefidz/information-theory/blob/master/books/Elements%20of%20Information%20Theory.pdf
https://arxiv.org/pdf/1611.02731
https://www.vetta.org/documents/Machine_Super_Intelligence.pdf
https://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://cs231n.github.io/
is_trueMay 2024
a backup just in case
https://nlp.seas.harvard.edu/annotated-transformer/
https://scottaaronson.blog/?p=762
https://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://arxiv.org/pdf/1409.2329.pdf
https://www.cs.toronto.edu/~hinton/absps/colt93.pdf
https://arxiv.org/pdf/1506.03134.pdf
https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
https://arxiv.org/pdf/1511.06391.pdf
https://arxiv.org/pdf/1811.06965.pdf
https://arxiv.org/pdf/1512.03385.pdf
https://arxiv.org/pdf/1511.07122.pdf
https://arxiv.org/pdf/1704.01212.pdf
https://arxiv.org/pdf/1706.03762.pdf
https://arxiv.org/pdf/1409.0473.pdf
https://arxiv.org/pdf/1603.05027.pdf
https://arxiv.org/pdf/1706.01427.pdf
https://arxiv.org/pdf/1611.02731.pdf
https://arxiv.org/pdf/1806.01822.pdf
https://arxiv.org/pdf/1405.6903.pdf
https://arxiv.org/pdf/1410.5401.pdf
https://arxiv.org/pdf/1512.02595.pdf
https://arxiv.org/pdf/2001.08361.pdf
https://arxiv.org/pdf/math/0406077.pdf
https://www.vetta.org/documents/Machine_Super_Intelligence.pdf
https://www.lirmm.fr/~ashen/kolmbook-eng-scan.pdf
https://cs231n.github.io/

What HN people wrote when they linked it

airgapstopgapMay 2023
Researchers aren't a monolith. Just because Gary Marcus (a complete fraud by the way, look up his "XProp" magitech he sold to Uber) pooh-poohed connectionist AI until ChatGPT came out doesn't mean nobody predicted gains from scaling; certainly many people deep in the know expected things to click. Certainly they expected that the basic trick of associative learning will be cracked. E.g. here's Shane Legg, December 2009 [1]:
> Conclusion: computer power is unlikely to be the issue anymore in terms of AGI being possible. The main question is whether we can find the right algorithms.
> One of the big things influencing me this year has been learning about how much we understand about how the brain works, in particular, how much we know that should be of interest to AGI designers. I won’t get into it all here, but suffice to say that just a brief outline of all this information would be a 20 page journal paper (there is currently a suggestion that I write such a paper next year with some Gatsby Unit neuroscientists, but for the time being I’ve got too many other things to attend to). At a high level what we are seeing in the brain is a fairly sensible looking AGI design. You’ve got hierarchical temporal abstraction formed for perception and action combined with more precise timing motor control, with an underlying system for reinforcement learning. The reinforcement learning system is essentially a type of temporal difference learning though unfortunately at the moment there…
nlJan 2014
It's difficult not to get caught up in the excitement around Deep Learning. The unsupervised nature of it at its best really does seem to approach human levels of learning (if not intelligence).
Also, I thought this was hilarious:
In a 2011 interview that predated DeepMind, co-founder Shane Legg said he gave only a 50 percent chance that human level-machine intelligence would exist by 2028.
ONLY?!? By 2028? That is a pretty radical prediction. Indeed, Legg says there is a 10% chance of human level intelligence by 2018, and 90% by 2050[1].
Legg has multiple papers published in the field of measuring machine intelligence, so I guess he has a pretty good view of the field.
If that 2028 prediction is even close to correct then I think setting up an ethics board now probably is the correct thing to do. (the notion that DeepMind asked that Google create an internal ethics board as a condition of the acquisition, as reported by The Information, had some AI researchers griping)
Edit: Legg's posting on his blog in 2011 deserves to be read:
I’ve decided to once again leave my prediction for when human level AGI will arrive unchanged. That is, I give it a log-normal distribution with a mean of 2028 and a mode of 2025, under the assumption that nothing crazy happens like a nuclear war. I’d also like to add to this prediction that I expect to see an impressive proto-AGI within the next 8 years. By this I mean a system with basic vision, basic sound processing, basic movement…
cute_boiJul 2026
No need stupid moving texts.
CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/
The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/
Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/
ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385
Multi-Scale Context Aggregation by Dilated Convolutions - https://arxiv.org/abs/1511.07122
Identity Mappings in Deep Residual Networks - https://arxiv.org/abs/1603.05027
Recurrent Neural Network Regularization - https://arxiv.org/abs/1409.2329
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin - https://arxiv.org/abs/1512.02595
Order Matters: Sequence to Sequence for Sets - https://arxiv.org/abs/1511.06391
Neural Machine Translation by Jointly Learning to Align and Translate - https://arxiv.org/abs/1409.0473
Pointer Networks - https://arxiv.org/abs/1506.03134
Attention Is All You Need - https://arxiv.org/abs/1706.03762
The Annotated Transformer - https://nlp.seas.harvard.edu/annotated-transformer/
Neural Turing Machines - https://arxiv.org/abs/1410.5401
A Simple Neural Network Module for Relational Reasoning - https://arxiv.org/abs/1706.01427
Relational Recurrent Neural Networks -…
cousin_itJun 2011
> If easy instances are enough, what's the relevance of the provably ideal uncomputable problem, besides rhetoric?
It shows that there's a path to AI that's gated on solving formal math problems instead of imitating wishy-washy emotional humans. This point may be obvious to you, but apparently it wasn't obvious to Stross.
> If humans used easy instances, and they're the only example we know of that achieved I, how is that not weak evidence that AI through easy instances will pass through human-like I?
Humans are also the only animals that can play chess, but chessplaying software didn't pass through a human-like stage where they got nervous, forgetful, etc. My intuition says math can't be that radically different from chess (both involve search on trees with a complicated evaluation function), and we're only a handful of new insights from having computers decisively beat humans at math. And at that moment, by implication from my previous point, they might just also beat us at everything else :-) Remember that, on an absolute scale, humans are about as bad at math as you can be and still build a civilization (otherwise civilization would've happened at a lower level of development) and also that we probably suck at math because math is a conscious task (see Moravec's paradox).
Related link that should make you scared if you buy my reasoning: http://www.vetta.org/2009/09/monte-carlo-aixi/ .
gooobJul 2026
https://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html
https://papers.nips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
https://arxiv.org/pdf/1512.03385
https://arxiv.org/pdf/1511.07122
https://arxiv.org/pdf/1603.05027
https://arxiv.org/pdf/1409.2329
https://arxiv.org/pdf/1512.02595
https://arxiv.org/pdf/1409.0473
https://arxiv.org/pdf/1506.03134
https://arxiv.org/pdf/1706.03762
https://nlp.seas.harvard.edu/annotated-transformer/
https://arxiv.org/pdf/1410.5401
https://arxiv.org/pdf/1706.01427
https://arxiv.org/pdf/1806.01822
https://arxiv.org/pdf/1704.01212
https://arxiv.org/pdf/2001.08361
https://arxiv.org/pdf/1811.06965
https://www.cs.toronto.edu/~hinton/absps/colt93.pdf
https://arxiv.org/pdf/math/0406077
https://scottaaronson.blog/?p=762
https://arxiv.org/pdf/1405.6903
https://onlinelibrary.wiley.com/doi/10.1002/047174882X.ch14
https://github.com/Bladefidz/information-theory/blob/master/books/Elements%20of%20Information%20Theory.pdf
https://arxiv.org/pdf/1611.02731
https://www.vetta.org/documents/Machine_Super_Intelligence.pdf
https://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://cs231n.github.io/
is_trueMay 2024
a backup just in case
https://nlp.seas.harvard.edu/annotated-transformer/
https://scottaaronson.blog/?p=762
https://karpathy.github.io/2015/05/21/rnn-effectiveness/
https://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://arxiv.org/pdf/1409.2329.pdf
https://www.cs.toronto.edu/~hinton/absps/colt93.pdf
https://arxiv.org/pdf/1506.03134.pdf
https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
https://arxiv.org/pdf/1511.06391.pdf
https://arxiv.org/pdf/1811.06965.pdf
https://arxiv.org/pdf/1512.03385.pdf
https://arxiv.org/pdf/1511.07122.pdf
https://arxiv.org/pdf/1704.01212.pdf
https://arxiv.org/pdf/1706.03762.pdf
https://arxiv.org/pdf/1409.0473.pdf
https://arxiv.org/pdf/1603.05027.pdf
https://arxiv.org/pdf/1706.01427.pdf
https://arxiv.org/pdf/1611.02731.pdf
https://arxiv.org/pdf/1806.01822.pdf
https://arxiv.org/pdf/1405.6903.pdf
https://arxiv.org/pdf/1410.5401.pdf
https://arxiv.org/pdf/1512.02595.pdf
https://arxiv.org/pdf/2001.08361.pdf
https://arxiv.org/pdf/math/0406077.pdf
https://www.vetta.org/documents/Machine_Super_Intelligence.pdf
https://www.lirmm.fr/~ashen/kolmbook-eng-scan.pdf
https://cs231n.github.io/

Shane Legg

Top posts on HN

What HN people wrote when they linked it

Shane Legg

Top posts on HN

What HN people wrote when they linked it

Top posts on HN

What HN people wrote when they linked it

Related

The Best Textbooks on Every Subject

There's No Fire Alarm for Artificial General Intelligence

Simplicity is hard. Let’s go shopping

Compound pejoratives on Reddit

AlphaFold: a solution to a 50-year-old grand challenge in biology

A look at Apple's technical approach to AI including core model performance etc.

Statistics vs. Machine Learning

Claude.md: Best Practices for Optimizing with Prompt Learning

Top posts on HN

What HN people wrote when they linked it

Related

The Best Textbooks on Every Subject

There's No Fire Alarm for Artificial General Intelligence

Simplicity is hard. Let’s go shopping

Compound pejoratives on Reddit

AlphaFold: a solution to a 50-year-old grand challenge in biology

A look at Apple's technical approach to AI including core model performance etc.

Statistics vs. Machine Learning

Claude.md: Best Practices for Optimizing with Prompt Learning