Preserve Knowledge
3 HN mentions · mentioned in 3 months, 2019–2024
22:51 · 26K views · Mar 7, 2018
Ali Rahimi accepts the NIPS 2017 Test of Time Award and argues that machine learning has become alchemy, with practice running ahead of understanding.
Cited when HN discusses the lack of rigor and theory in deep learning, often with a pointer to the key moment of the talk.
3 comments from Hacker News
"There is no such thing as not worshipping. Everybody worships. The only choice we get is what to worship. And the compelling reason for maybe choosing some sort of god or spiritual-type thing to worship–be it JC or Allah, be it YHWH or the Wiccan Mother Goddess, or the Four Noble Truths, or some inviolable set of ethical principles–is that pretty much anything else you worship will eat you alive. If you worship money and things, if they are where you tap real meaning in life, then you will never have enough, never feel you have enough. It’s the truth. Worship your body and beauty and sexual allure and you will always feel ugly. And when time and age start showing, you will die a million deaths before they finally grieve you. On one level, we all know this stuff already. It’s been codified as myths, proverbs, clichés, epigrams, parables; the skeleton of every great story. The whole trick is keeping the truth up front in daily consciousness." - DFW Academic science resembles religion with its dogmas, nepotism, bureaucracies, and favor-currying shibboleths. Deep learning in particular is akin to modern alchemy [1]. [1] this video
> It’s logically obvious. You can recreate any linear regression using a neural network. So a neural network approach will always be at least as good as linear regression. Well, maybe it's logically obvious, but it's also empirically false: this video (Skip to 13:36 to get to the point) > But a neural network can model non-linear relationships as well. Now ask yourself, what is the likelihood of there being a non-linear relationship in the data? As with most real-world data, the likelihood of non-linear relationships is extremely high. That’s why we can be quite certain a neural network approach will able to outperform linear regression. As I've said elsewhere in this thread, a deep neural network of N layers can be decomposed into a complex nonlinear function of N - 1 layers followed by a linear combination of the nonlinear features generated that function. There's no law that says that you have to use a neural network to generate your nonlinear features. You can use any method you like and then linearly combine those.
Machine Learning is Alchemy this video . We need to understand optimization to advance this field not train thousands of monkeys to press play and hope for a nice result. Also, why did they not decide to teach 1% of the population basic coding. Now that could transform a society.