Google Research Blog

> And is it going to be more successful because it is faster?

There's a lot of research out there about the link between page performance and user retention rate. And this makes sense: If newegg is taking forever to browse, I'll switch to amazon and newegg loses out on a decent chunk of change.

https://research.googleblog.com/2009/06/speed-matters.html

So, up to a point, yes, yes they are going to be more successful because it is faster. 200ms on my broadband connected desktop isn't that much, but Google is able to measure it's impact. And that might be a second or two on my cellular connected phone.

> Is it maintainable.

A lot of optimizations I've seen involve simplifying stuff. Fast and maintainable don't have to be at odds. I wouldn't care to guess for the whole, but, for example, do you really think using system fonts instead of embedding your own is more complicated, harder to maintain, and more work? I doubt it, and that's one of the optimizations suggested.

Now, yes, with optimizing, there is a break even point where it's no longer worth it to push further, but it's also not necessarily obvious where that is if you're just taking it a task at a time. Keep in mind: some of this is research for effective and ineffective techniques for optimizing other websites, and evaluating which ones are maintainable (or not) for future projects. To know what to bother with and what not to bother with when implementing the rest of the codebase. If you're just worried…

Another recent "super resolution" method (RAISR) from Google Research:

https://arxiv.org/abs/1606.01299

https://research.googleblog.com/2016/11/enhance-raisr-sharp-images-with-machine.html

>Given an image, we wish to produce an image of larger size with significantly more pixels and higher image quality. This is generally known as the Single Image Super-Resolution (SISR) problem. The idea is that with sufficient training data (corresponding pairs of low and high resolution images) we can learn set of filters (i.e. a mapping) that when applied to given image that is not in the training set, will produce a higher resolution version of it, where the learning is preferably low complexity. In our proposed approach, the run-time is more than one to two orders of magnitude faster than the best competing methods currently available, while producing results comparable or better than state-of-the-art.

>A closely related topic is image sharpening and contrast enhancement, i.e., improving the visual quality of a blurry image by amplifying the underlying details (a wide range of frequencies). Our approach additionally includes an extremely efficient way to produce an image that is significantly sharper than the input blurry one, without introducing artifacts such as halos and noise amplification. We illustrate how this effective sharpening algorithm, in addition to being of independent interest, can be used as a pre-processing step to induce the learning of more effective upscaling…

A piece of more recent history that people might not know: the 32-bit SPARC chips had some special instructions for adding and subtracting integers, assuming a representation in which the two low-order bits were used as tag bits, and a value of 00 in those bits indicated a fixnum (Lisp-speak for an immediate integer). The TADDCC instruction ("Tagged ADD and set Condition Codes") was just like ADD, except that it would also set the overflow condition code if either operand had a tag value other than 00 (it would also be set if the addition overflowed). The next instruction would call a handler if the overflow bit was set; the handler would determine whether (a) this was a case of actual integer overflow requiring construction of a "bignum" (a heap-allocated integer); (b) one or both operands were already heap-allocated integers (or perhaps other kinds of numbers) requiring some other form of addition; or (c) one or both operands were not numbers at all, a runtime error. But the point is, the most common case was handled in two instructions, and those could even be combined into one using the TADDCCTV variant, which trapped automatically on overflow. (There were also TSUBCC and TSUBCCTV, of course.)

My understanding of the history here is that a lot of Sun people, notably Bill Joy, came from Berkeley, where Franz Lisp was also developed; and in the mid-1980s (the first SPARC came out in 1987) it still seemed possible that Lisp would be of great commercial importance. …

The article is riddled with errors that undermine its own thesis.

It starts badly:

> Artificial Intelligence is colossally hyped these days, but the dirty little secret is that it still has a long, long way to go

This is not a secret, let alone a dirty one. Even 5 minutes casual research into the state of AI will reveal what it can do and what it can't.

It says:

> Such systems can neither comprehend what is going on in complex visual scenes (“Who is chasing whom and why?”) nor follow simple instructions (“Read this story and summarize what it means”).

In fact comprehension of (very) simple stories is now more or less a solved problem. I wrote about performance on the bAbI tests here:

https://blog.plan99.net/the-science-of-westworld-ec624585e47

Summarisation of stories is also something with good recent results:

https://research.googleblog.com/2016/08/text-summarization-with-tensorflow.html

Summarisation of arbitrary video is harder but given that object and path extraction already works well, it doesn't seem very implausible that we'll see some good research results in video summarisation systems within a few years. Extrapolation from what's happening to hypothesised explanations is a lot harder but not hard to imagine it being possible given the direction research is going.

> My daughter had never seen anyone else disembark in quite this way; she invented it on her own. Presumably, my daughter relied on an implicit theory of how her body moves, along with an…

Yeah, I'm confused that this is the top comment; it's factually incorrect. NASNet is an example of a result of AutoML. To quote the Google blogpost on NASNet:

>In Learning Transferable Architectures for Scalable Image Recognition, we apply AutoML to the ImageNet image classification and COCO object detection dataset... AutoML was able to find the best layers that work well on CIFAR-10 but work well on ImageNet classification and COCO object detection. These two layers are combined to form a novel architecture, which we called “NASNet”.

[https://research.googleblog.com/2017/11/automl-for-large-scale-image.html, November 2017]

In contrast AutoML is, as the nytimes article describes, "a machine-learning algorithm that learns to build other machine-learning algorithms". More specifically, from the Google blogpost about AutoML:

>In our approach (which we call "AutoML"), a controller neural net can propose a “child” model architecture, which can then be trained and evaluated for quality on a particular task...Eventually the controller learns to assign high probability to areas of architecture space that achieve better accuracy on a held-out validation dataset, and low probability to areas of architecture space that score poorly.

[https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html, May 2017]

Quoc, Barret, and others have been working on ANN-architecture-design systems for a while now (see: https://arxiv.org/abs/1611.01578), and AutoML…

(The comment below incorporates an edited version of my comment from last week's thread.)

With human involvement only at the meta level, deep understanding of the generated implementations becomes more challenging and, in highly complex domains, perhaps impossible. See AutoML's generated architecture and compare that with a human-designed one. [1] [2]

A future advance could allow the machines to automatically pick and learn to do new tasks that are helpful to accomplish a given high-level mission. Therefore, chances of unintended consequences become much greater.

The major issue is, without a moral core that closely aligns with humanity's evolved morality, there will be moves that advanced AIs come up with that we deem abhorrent, and sometimes unforeseeable, yet they perform them innocently and we only find out the consequences once it is too late.

Microsoft's Tay tweet debacle was fairly inconsequential [3]. But AI is becoming more powerful and granted more and more authority over the real world we live in, can we afford the technology without humanity?

Developing a moral core should be prioritized.

[1] Google Blog on AutoML https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html

[2] Many tasks in the real world are more complex than Go and even expert humans are not capable of understanding all possible complex interactions on the Go board.

See: AlphaGo, in context by Andrej Karpathy…