GoogleDeveloperDay
3 HN recommenders · recommended in 4 months, 2007–2009
52:52 · 27K views · Jun 5, 2007
Peter Norvig's 2007 Google talk on theorizing from data, covering what huge datasets can do for language understanding, translation, information extraction and inference. It argues that more data often beats cleverer models.
HN recommends it as the clear explanation of why Google's statistical, data-first approach works.
5 comments from Hacker News
One talk in particular in which he focuses on this idea, including with facts and figures from research literature, is a talk he gave a while ago called "Theorizing from Data": this video . You should definitely watch it if you haven't seen it before. Norvig states his opinion slightly differently: of course being smart about algorithms is good. But until you get a lot of it, you often can't even fairly evaluate different algorithms. It's only when you're no longer getting significant gains from more data that you should then start thinking about being an algorithm smartypants.
[ this video ] Its a good talk, though maybe should be called "theorizing from massive amounts of data". Makes you wonder what Google are keeping under wraps for now, and how the powerset approach can compete, except maybe for domain-specific stuff.
If you really think that's all they're doing then you have very little understanding in just how subtly awesome some of the stuff they're doing is. Peter Norvig does a better job explaining some of it than I ever could. this video
Humans resolve a lot of the complexity intuitively and unconsciously. It is not an inherently bad way to handle it, just incompatible with current programming paradigms. This is not to discount the value of clear explicit thinking in humans, it is just to say that programming would benefit a lot from ability to apply common sense to mundane things. For some dramatic evidence that imprecise is better and it really works, look at the success of Google and the non-delivery of the Semantic Web. Google's approach is probably best explained by Peter Norvig in his "Theorizing from Data" lecture: this video State of the art in natural language processing is achieved with statistical inference, not building precise linguistic models. See e.g. Marti Hearst's text here: [LINK] Back to programming languages, I believe that a lot of the Ruby/Rails success is due to mimicking natural language in DSLs. Rails goes as far as using pluralization. DSLs bring convenience at the price of less understanding: people tend to try using them intuitively, without understanding precisely what happens, much more so than they do with libraries/frameworks. Perl, similarly, was created with linguistic principles in mind: [LINK] It is by no means conclusive evidence, but I believe there is an underlying trend.
I like that they show you the source pages, confidence, and alternative sources for the data. However, it suffers the classic Google app problem; it does interesting stuff, but just looks ugly, and is generally unappealing. Hard for me to explain why, but despite my interest in the technology I just didn't enjoy using it at all. But it is interesting stuff, I'd been wondering when something like this would emerge since watching this Norving talk from 2007 [ this video ].