The success of most deep learning algorithms today is largely the result of decades of research, the growing availability of GPUs, and data. But not just any kind of data — the kind that is abundant, clean, and labeled.

Datasets like ImageNet, CIFAR10, SVHN, and others, have allowed researchers and practitioners to make remarkable progress on computer vision tasks and were immensely useful for our own experimentation. Yet the elephant in the room for many applications that seek to benefit from this progress, such as medicine, is precisely the fact that the data must be abundant, clean, and labeled.

**Semi-supervised learning (SSL), **a subfield that combines both supervised and unsupervised learning, has grown in popularity in the deep learning research community over the past few years. It’s very possible that, at least in the short-term, SSL approaches could be the bridge between label-heavy supervised learning and a future of data-efficient modeling.

In this post, we talk about when you should consider using SSL approaches in your production environments and the lessons we’ve learned using them to improve our object detection models at Uizard. Of course, we’ll do our best to share the big picture but keep some details of the wizardry to ourselves.

Our hope is that by displaying how and when SSL worked and didn’t work for us and by sharing tips learned on our journey from research to production, we can inspire you to take a chance on SSL for your work and unlock the potential of your unlabeled data.

In short, here are a few lessons we emphasize:

• Simplicity is king. The most successful approaches in SSL that translated from research to production were those that were the simplest to reproduce. Specifically, we’ll elaborate on how “Self-Training with Noisy Student” (Xie et al., 2019) worked for us.
• Pseudo-label refinement with heuristics can be extremely effective. Pseudo-labeling is a popular component of SSL approaches — we find that using simple heuristics to refine our pseudo-labels in the unlabeled data improves performance across different sizes of unlabeled datasets.
• **Progress in semi-supervised image classification is difficult to translate to object detection. **Much of the progress in SSL that we followed measured performance on image classification with promises of similar improvements on object detection, but we found it difficult to adapt them appropriately in practice. As a result, more work and research is needed in the semi-supervised object detection space.

#production #semi-supervised-learning #machine-learning #research #deep-learning