Christopher Hesse writes about machine learning image projects, mainly a TensorFlow port of pix2pix and the edges2cats demo.
HN links the edges2cats demo as an example of image-to-image translation and as a pointer for people wanting to try pix2pix-style models.
One very practical reason is to save on computation costs. GPU servers are not cheap and if you're going to run on CPU anyway, you might as well run it locally on the user's browser.
Some demos such as real time object detection isn't possible at all if you had to pay roundtrip server latency (not to mention the complexity of streaming video to and from your server): https://github.com/ModelDepot/tfjs-yolo-tiny
And a lot of web demos such https://affinelayer.com/pixsrv/ I don't believe will be up for such a long time if the author had to pay ongoing server costs.
yeah lot of these "AI does the thing" stuff looks extremely interesting to play with, especially to see what happens when you poke the model with weird data (say, no eye, no hair, two noses etc) but most of them are completely inaccessible to non researchers, even when the code/model is released building the solution requires so much effort and expertise to make it impossible to play with
this is why I loved so much the pix2pix cat drawing demo https://affinelayer.com/pixsrv/ I hoped it would make a turning point for demoes but alas this is still unique
You never saw edges2cats I take it? https://affinelayer.com/pixsrv/
> I don't understand how the edges-to-faces can possibly work. The inputs seem to be black & white, and yet the output pictures have light skin tones.
The step you're missing is that an edge detector is run on the entire database of training images to produce a database of edge images. The input edge image is run against that corpus of edge images in order to find which edge images match, then sample the corresponding original color images and synthesize a new color image.
> This image from the article shows that the original image and the fantasy image are not alike at all.
This is another avenue that could be further explored, which I quite like. That is, a non-artist can doodle images and create a completely new photo-realistic image based on the line drawings.
I was modifying a few images (from link on another comment here: https://affinelayer.com/pixsrv/ ) and the end results were interesting.
You'll probably want something closer to a GAN like pix2pix - https://phillipi.github.io/pix2pix/
An example implementation would look something like edges2cats https://affinelayer.com/pixsrv/
The live demo is super fun (I really hope this does not kill it) : https://affinelayer.com/pixsrv/