Stop using Image Interpolation for Neural Audio Synthesis

Stop using Image Interpolation for Neural Audio Synthesis

In this story I want to advance your current understanding of neural upsamplers in the context of audio synthesis

The different types of Neural Upsamplers and which one you should use in your Deep Learning Audio Synthesis Project.

In this story I want to advance your current understanding of neural upsamplers in the context of audio synthesis. And provide a simple Subpixel1D Keras layer implementation to use as a drop-in replacement for many of the tasks we discuss today.

We all know that up- and down sampling is an important operation in deep learning for computer vision, e.g., in tasks like image super resolution or image generation. The same holds true for audio synthesis using popular architectures like GANs, U-Nets or Auto-encoder. While downsampling is a relatively simple operation, there always have been difficulties finding a good upsampling strategy which doesn’t result in image or audio artifacts. For a primer on 2-dimensional checkerboard artifacts in computer vision tasks read this great post [1].

Now let us dive deeper into 1-dimensional audio upsampling. In the audio domain we use three main upsampling techniques [2]:

  1. Transposed convolutions (widely used)
  2. Interpolation + convolution (often used)
  3. Subpixel convolutions (rarely used but prominent in vision tasks)

Examples of their usage can be found in many publications, like, Demucs (music source separation) [3], MelGAN (waveform synthesis) [4], SEGAN (speech enhancement) [5], Conv-TasNet (speech separation) [6] or Wave U-Net (source separation) [7].

TensorFlow Keras provides a fourth solution to upsampling which is the UpSample1D layer, however, as of now (March 21) this layer still is outrageously slow on GPU, although the issue is closed.

deep-learning audio editors-pick tensorflow machine-learning

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Machine Learning Vs Deep Learning: Difference Between Machine Learning and Deep Learning

This article will simply explain the concept which will help you understand the difference between Machine Learning and Deep Learning. 

Hire Machine Learning Engineer | Offshore Machine Learning Experts

We are a Machine Learning Services provider offering custom AI solutions, Machine Learning as a service & deep learning solutions. Hire Machine Learning experts & build AI Chatbots, Neural networks, etc. 16+ yrs & 2500+ clients.

Top Deep Learning Development Services | Hire Deep Learning Developer

Inexture's Deep learning Development Services helps companies to develop Data driven products and solutions. Hire our deep learning developers today to build application that learn and adapt with time.

5 Latest Technology Trends of Machine Learning for 2021

Check out the 5 latest technologies of machine learning trends to boost business growth in 2021 by considering the best version of digital development tools. It is the right time to accelerate user experience by bringing advancement in their lifestyle.

Hire Machine Learning Developers in India

We supply you with world class machine learning experts / ML Developers with years of domain experience who can add more value to your business.