GANs (Generative Adversarial Networks) have taken the world of deep learning and computer vision by storm since they were introduced by Goodfellow et al. in 2014 at NIPS. The main idea of GANs is to simultaneously train two models; a generator model G that captures a certain data distribution, and another discriminator model D that determines whether a sample came from the original distribution or from G.

The GAN framework is like a two player min-max game. **_G _**continually improves to generate images that are more realistic and have better quality. D improves in its ability to determine whether an image was created by G. Training a GAN can be done completely with backpropagation, which highly simplifies the training process. Typically, training is performed by regular switching from G to D in order to prevent a huge performance gap in the two models.

Image Restoration

To explain GANs in more detail, we will use the example of image restoration, using the code from Lesson 7 of course-v3 from fast.ai. You can make a copy of the notebook on Google Colab and run the code yourself while reading through for a more hands-on experience! An advantage is that we only need an unlabeled dataset of images to create an image restoration model. The aim of this model is to restore a low resolution image and remove simple watermarks. Here is a brief overview of the image restoration process:

1. Decide on the dataset to use. In this post we will be using the Oxford-IIIT Pet Dataset, which is publicly available under a CC 4.0 License.
2. ‘Crappify’ the dataset by performing certain transformations to the images.
3. Pre-train a generator network with a UNet architecture to transform crappified images back into the original images.
4. Generate an initial set of restored images that can be used to pre-train the critic.
5. Pre-train a critic network to classify generated images as ‘fake’ and original images as ‘real’.
6. Train the entire GAN structure, switching from Generator to Critic as per the GAN paper.
7. Finally, we will obtain a generator network that can be used to restore other images that are of lower quality!

#image-processing #deep-learning #machine-learning #generative-adversarial