Is it possible to do text-classification with 150 target classes using only 10 labelled samples for each class but still get a good performance?
Starting from that simple question, I start to do research in order to answer that question. After spending several hours, I ended up with GAN-BERT. What is GAN-BERT? What experiment that I did using GAN-BERT? In this article, I will try to give a brief introduction of GAN-BERT and also the implementation of it for Intent Classification using CLINC150 Dataset.
In Natural Language Processing (NLP) field, BERT or Bidirectional Encoder Representations from Transformers is a well-known technique based on Transformers architecture to do a wide range of tasks, including text classification. However, this technique can perform well when there is ‘enough’ labelled training data to be exploited while obtaining labelled data is time-consuming and a costly process. The potential solution for this is using a semi-supervised learning approach.
Semi-supervised learning is an approach in machine learning field which combines both labelled and unlabelled data during training. The goal is the same as the supervised learning approach, that is to predict the target variable given the data with several features. This approach is crucial when we have not so many labelled data while our model needs a lot of training data to perform well.
Recently in July 2020, a paper named “GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples”, try to extend the fine-tuning of BERT-like architectures with unlabeled data in a generative adversarial setting. In high-level, they try to enrich the BERT fine-tuning process with an SS-GAN (Semi-supervised GAN) perspective.
“In this paper, we extend the BERT training with unlabeled data in a generative adversarial setting. In particular, we enrich the BERT fine-tuning process with an SS-GAN perspective, in the so-called GAN-BERT model”
#nlp #naturallanguageprocessing #data-science #text-classification #machine-learning #deep learning
