Autoencoders — Deep Learning bits #1

: data compression, image reconstruction and segmentation (with examples!) Featured In the “ ” series, we will see how to use deep learning to solve complex problems end-to-end as we do in We will rather look at different techniques, along with some Deep Learning bits not A.I. Odyssey . examples and applications. If you like Artificial Intelligence, make sure to subscribe to the newsletter to receive updates on articles and much more! Introduction What’s an autoencoder? Neural networks exist in all shapes and sizes, and are often characterized by their and For instance, image classifiers are built with They take as inputs, and output a . input output data type. Convolutional Neural Networks. images probability distribution of the classes are a family of neural networks for which . They work by compressing the input into a and then reconstructing the output from this representation. Autoencoders (AE) the input is the same as the output* latent-space representation, *We’ll see how using altered versions of the input can be even more interesting Simple Autoencoder architecture — The input is compressed and then reconstructed Convolutional Autoencoders A really popular use for autoencoders is to apply them to images. The is to replace layers by layers. These, along with pooling layers, convert the input from and (let’s say 100 x 100 px with 3 channels — RGB) to and . This helps the network extract from the images, and therefore obtain a much more accurate latent space representation. The reconstruction process uses and convolutions. trick fully connected convolutional wide thin narrow thick visual features upsampling The resulting network is called a ( ). Convolutional Autoencoder CAE Convolutional Autoencoder architecture — It maps a wide and thin input space to narrow and thick latent space Reconstruction quality The of the input image is often and of This is a consequence of the during which we have . reconstruction blurry lower quality. compression lost some information The CAE is trained to reconstruct its input The reconstructed image is blurry Use of CAEs Example 1: Ultra-basic image reconstruction Convolutional autoencoders can be useful for reconstruction. They can, for example, learn to , or reconstruct missing parts. remove noise from picture To do so, we don’t use the same image as input and output, but rather a and the . With this process, the networks learns to fill in the gaps in the image. noisy version as input clean version as output Let’s see what a CAE can do to . . We can manually create the dataset, which is extremely convenient. replace part of an image of an eye Let’s say there’s a crosshair and we want to remove it The CAE is trained to remove the crosshair Even though it is blurry, the reconstructed input has no crosshair left Now that our autoencoder is trained, we can use it to remove the crosshairs on pictures of eyes we have never seen ! Example 2: Ultra-basic image colorization In this example, the CAE will learn to from an image of circles and squares to the same image, but with the colored in , and the in . map circles red squares blue The CAE is trained to colorize the image Even though the reconstruction is blurry, the color are mostly right The CAE does on the parts of the image. It has understood that are and are . The color comes from a blend of blue and red where the networks between a circle and a square. pretty well colorizing right circles red squares blue purple hesitates Now that our autoencoder is trained, we can use it to colorize pictures we have never seen before ! Advanced applications The examples above are just to show what a convolutional autoencoder can do. proofs of concept More exciting application include , , or . The latter is obtained by using the low resolution as input and the high resolution as output. full image colorization latent space clustering generating higher resolution images Colorful Image Colorization by Richard Zhang, Phillip Isola, Alexei A. Efros Neural Enhance by Alexjc Conclusions In this post, we have seen how we can use to compress, reconstruct and clean data. Obtaining images as output is something really thrilling, and to play with. autoencoder neural networks really fun Note: there’s a modified version of AEs called Variational Autoencoders , which are used for image generation, but I keep that for later. If you like Artificial Intelligence, make sure to subscribe to the newsletter to receive updates on articles and much more! You can play with the code over there: _ConvolutionalAutoencoder - Quick and dirty example of the application of convolutional autoencoders in Keras/Tensorflow_github.com despoisj/ConvolutionalAutoencoder Thanks for reading this post, stay tuned for more !