LessWrong

> I think I'm not being out of line to claim that on the average an American's high-school-level education is pretty poor.

The thing is that it's not really that true. Here's a typical article on Huffington Post bemoaning the American education system: http://www.huffingtonpost.com/2012/07/23/us-students-still-lag-beh_n_1695516.html

But if you read it carefully, the truth is that we're not getting worse. We're just not getting better as fast as others. We used to be the best. Now we're in the middle. We're not worse than we were 60 years ago. We're better. Other countries have just gotten better much faster. Which is a trend that applies to a lot in America these days.

Education IS improving. So, I think anything that tries to relate an absolute decline to "the failure of our schools" is wrong. Maybe that's not your point though.

The thing is that what you describe at the beginning of your post is actually typically human. Experiments have shown we all have cognitive biases that make us bad at "being skeptical" and "reasoning about potential outcomes" or using past examples to predict future events (even when we're well educated.) Humans are just not good at doing that stuff. We can overcome our biases but that's not really taught in school. You can read about a lot of it at http://lesswrong.com/

So I don't think it's that helpful to bundle up people who struggle with cognitive biases as "Those kinds of people" There's a chance that "our kind of people",…

Just like HN grew around the writing of Paul Graham, the "rationalist community" grew around the writings of Eliezer Yudkowsky. Similar to how Paul Graham no longer participates on HN, Eliezer rarely participates on http://lesswrong.com anymore, and the benevolent dictator for life of lesswrong.com is someone other than Eliezer.

Eliezer's career has always been centered around AI. At first Eliezer was wholly optimistic about AI progress. In fact, in the 1990s, I would say that Eliezer was the loudest voice advocating for the development of AI technology that would greatly exceed human cognitive capabilities. "Intentionally causing a technological singularity," was the way he phrased it in the 1990s IIRC. (Later "singularity" would be replaced by "intelligence explosion".)

From 2001 to 2004 he started to believe that AI has a strong tendency to become very dangerous once it starts exceeding the human level of cognitive capabilities. Still, he hoped that before AI starts exceeding human capabilities, he and his organization could develop a methodology to keep it safe. As part of that effort, he coined the term "alignment". The meaning of the term has broadened drastically: when Eliezer coined it, he meant the creation of an AI that stays aligned with human values and human preferences even as its capabilities greatly exceed human capabilities. In contrast, these days, when you see the phrase "aligned AI", it is usually being applied to an AI system that is not a threat to…

Came here to say this.

This kind of work irks me. No offense to the author, this is a widespread trend, not a single example.

You write a whole blog-post on a topic and not mention Hanlon's razor. It is inexcusable that you didn't know about it. It was your job to do "literature survey" or some google search, or an intellectual discussion with a handful of people, (ideally all three of them, and more) before you blurt out a whole essay on a topic. And then in the essay you create an image of yourself originally coming up with such a brilliant idea.

This is just a piece of writing (and maybe I am a bit harsh but I'm just trying to point to a trend). We're on HN. This is way worse when it comes to coding/github projects:

You create some kind of code, put it on github, create a blog-post in some shiny flat/material design, grey font, maybe medium or nautil.us posting. And put that posting on tech forums, HN, linkedin, reddit, whatever.

And you fail to mention other projects preceding you on this same exact topic, much less discuss the pros and cons of your approach vs previous approach(es). Who knows you might've ended up finding out your own code was a complete reinventing of the wheel and completely redundant (maybe you're too afraid to find that out). (As an example, look at my recent comment about 'building your own linux' vs LFS [1]).

Creating something new and original is hard work. Part of that hard work is searching for existing efforts, related work, doing a…

I think it makes sense to debate whether this pause can or cannot work and the game theory behind it.

But to those that trivializes the potential danger of AI and compare it to the likes of climate change is ignorant at best. AI has been consistently ranked as the highest potential cause for the extinction of the entire human race (by experts, a random survey of the population will likely say climate change, sadly). (https://riskfrontiers.com/insights/ranking-of-potential-causes-of-human-extinction/)

Human caused climate change at the worst potential possible outcome would not even make a dent to the human population at a multi-century time-scale. We also have a lot of solutions in place for climate change that merely requires time and investment for adoption. Climate change's impact to humanity is less than the alarming birth rate we are seeing today (which also, we have no solution for...)

We literally have no solution for AI Alignment, and this is with a lot of smart people working on it. From Bostrom to Eliezer and many others (check https://lesswrong.com). A lot of experts have give up because they see no way out and is only merely thinking about how should humanity end with dignity. We can't even figure a way out of the prisoner dilemma on doing a 6 months pause.

Comparing AI Alignment with Climate Change trivializes the matter and only illustrate the ignorance of the populace. Another reason why we will not solve AI Alignment and will likely go extinct.

There are many methods, but this is one of my favorites.

If the bias is rooted in fear, then it can sometimes help to ask yourself a bunch of "what if X" questions, where X can be anything related to the idea. While doing this, pay close attention for any subtle feelings of fear, anxiety, excitement, or any other emotion. If they are present, then that's a hint that your emotional biases may be in play.

For example, I believe climate change is real. So I ask myself, "What if climate change isn't real?" I get a little flare of fear, so I know that I am at least partially motivated by an emotional bias there.

Once you detect any emotional bias, that's when you bring Rationality skills into play and start using the scientific method. Test your beliefs. Ask yourself, "What's the easiest way to disprove my own belief?" and then perform that test or do that research. If you give it honest effort, and your belief stands up to many tests, then you can be more confident in it.

It can be tricky to be mindful enough to detect those subtle traces of fear, but with practice it gets easier.

If you want to get even better at working with your biases, check out the broader Rationality topic [0], and perhaps read some rationality fiction [1], it's amazing stuff.

[0] https://www.lesswrong.com/

[1] https://www.hpmor.com/

For those truly interested in improving their rationality, http://www.lesswrong.com is in a similar format to Hacker News and is dedicated to the subject.

More directly addressing this article, K.E. Stanovich has conducted some excellent research showing relationships between cognitive abilities and thinking styles. It is fully available online. General reasoning/rationality works are available here: http://web.mac.com/kstanovich/iWeb/Site/Research%20on%20Reasoning.html

Here's a particularly relevant piece: http://web.mac.com/kstanovich/iWeb/Site/Research%20on%20Reasoning_files/jexppsy98.pdf (Various cognitive bias tests are conducted to explore relations between cognitive functions and challenge assumptions on normative models of rationality. Really interesting exploration of competence-performance dichotomy with focus on effects of thinking dispositions vs. algorithmic limitations. The effect of thinking disposition in achieving normative rationality is an important consideration that applies to considerations for improving your own thought/action models)