This article is part of our [reviews of AI research papers], a series of posts that explore the latest findings in artificial intelligence.

Two things often mentioned with deep learning are “data” and “compute resources.” You need a lot of both when developing, training, and testing deep learning models. When developers don’t have a lot of training samples or access to very powerful servers, they use transfer learning to finetune a pre-trained deep learning model for a new task.

At this year’s ICML conference, scientists at IBM Research and Taiwan’s National Tsing Hua University Research introduced “black-box adversarial reprogramming” (BAR), an alternative repurposing technique that turns a supposed weakness of deep neural networks into a strength.

BAR expands the original work on adversarial reprogramming and previous work on black-box adversarial attacks to make it possible to expand the capabilities of deep neural networks even when developers don’t have full access to the model.

Pretrained and finetuned deep learning models

When you want to develop an application that requires deep learning, one option is to create your own neural network from scratch and train it on available or curated examples. For instance, you can use ImageNet, a public dataset that contains more than 14 million labeled images.

There is a problem, however. First, you must find the right architecture for the task, such as the number and sequence of convolution, pooling, and dense layers. You must also decide the number of filters and parameters for each layer, the learning rate, optimizer, loss function, and other hyperparameters. A lot of these decisions require trial-and-error training, which is a slow and costly process unless you have access to strong graphics processors or specialized hardware such as Google’s TPU.

To avoid reinventing the wheel, you can download a tried-and-tested model such as AlexNet, ResNet, or Inception, and train it yourself. But you’ll still need a cluster of GPUs or TPUs to complete the training in an acceptable amount of time. To avoid the costly training process, you can download the pre-trained version of these models and integrate them into your application.

Alternatively, you can use a service such as Clarifia and Amazon Rekognition, which provide application programming interfaces for image recognition tasks. These services are “black-box” models because the developer doesn’t have access to the network layers and parameters and can only interact with them by providing them images and retrieving the resulting label.

Now, suppose you want to create a computer vision algorithm for a specialized task, such as detecting autism from brain scans or breast cancer from mammograms. In this case, a general image recognition model such as AlexNet or a service like Clarifai won’t cut it. You need a deep learning model trained on data for that problem domain.

#blog #adversarial attacks #deep learning #machine learning