Style transfer is a very popular deep learning task that lets you change an image’s composition by applying the visual style of another image.

From building artistic photo editors to giving a new look to your game designs through state-of-the-art themes, there are plenty of amazing things you can build with neural style transfer models. It’s also can be handy or data augmentation.

At WWDC 2020, Create ML (Apple’s model building framework) got a big boost with the inclusion of style transfer models. Though it’s shipped with the Xcode 12 update, you’ll need macOS Big Sur (in beta at the time of writing) to train style transfer models.

A First Look at Create ML’s Style Transfer

Create ML has now unlocked the potential to train style transfer models right from your MacBook. You can train both image and video style transfer convolutional neural networks, with the latter using only a limited set of convolution filters to make it optimized for real-time image processing.

To get started, you need three things:

• A styling image (also called a style reference image). Typically, you can use famous paintings or abstract art images to let your model learn and impart the style from. In our case, we’ll be using a pencil sketch image in our first model (check the screenshot below).
• A validation image that helps visualize model quality during the training process.
• Dataset of content images that act as our training data. To get optimal results, it’s best to work with a directory of images similar to ones you’d use while running inference.

#ios #ios-app-development #machine-learning #heartbeat #programming