Review: High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis

Review: High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis

Review: High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis. We will dive into another inpainting method which can be regarded as an improved version of Context Encoders.

Helloooooo guys! In the previous post, we have gone through the introduction to image inpainting and the first GAN-based inpainting algorithm, Context Encoders. If you have not read the previous post, I highly recommend you to have a quick look of it first! This time, we will dive into another inpainting method which can be regarded as an improved version of Context Encoders. Let’s start!

Recall

Here, I briefly recall what we have learnt in the previous post.

  • Deep semantic understanding of an image or the context of an image is important to the task of inpainting, and (channel-wise) fully-connected layer is one way to capture the context of an image.
  • For image inpainting, visual quality of the filled images is much more important than the pixel-wise reconstruction accuracy. More specifically, as there is no model answer to generated pixels (we do not have the ground truth in real-world situations), we just want look-realistic filled images.

Motivation

  • Existing inpainting algorithms can only handle low-resolution images because of the memory limitations and the training difficulty in high-resolution images.
  • Although state-of-the-art inpainting method, Context Encoders, can successfully regress (predict) the missing parts with certain degree of semantic correctness, there is still room for improvement in the textures and details of the predicted pixels as shown in Figure 1.

Introduction

  • Context Encoder is not perfect. i) texture details of the generated pixels can be further improved. ii) not able to handle high-resolution images.
  • At the same time, Neural Style Transfer is a hot topic in which we would like to transfer the style of an image (style image) to another image with its same content (content image) as shown in Figure 2 below.

Image for post

Figure 2. Example to illustrate the task of style transfer [2]

  • Note that textures and colors can be regarded as a kind of styles. The authors of this paper employ the style transfer algorithm to enhance the texture details of the generated pixels.

convolutional-network image-inpainting generative-adversarial deep-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Applying Generative Adversarial Network to generate novel 3D images

In this post, I’ll demonstrate the behavior of Generative Adversarial Networks (GANs) on 3D images and how it can help to generate novel 3D images.

Implementing Deep Convolutional Generative Adversarial Networks

Deep Convolutional Generative Adversarial Networks or DCGANs are the ‘image version’ of the most fundamental implementation of GANs.

Revision for Deep Image Inpainting and Review: Patch-Based Image Inpainting

Today, I would like to give a revision for deep image inpainting we have talked about so far. Also, I want to have another review of an image inpainting paper for the consolidation of knowledge of deep image inpainting.

Decoding the Science Behind Generative Adversarial Networks

A dummies guide to GANs that aims at art of faking! Generative adversarial networks(GANs) took the Machine Learning field by storm last year with those impressive fake human-like faces.

Deep learning on graphs: successes, challenges, and next steps

Deep learning on graphs: successes, challenges, and next steps. TL;DR This is the first in a series of posts where I will discuss the evolution and future trends in the field of deep learning on graphs.