Introduction
This article is aimed at giving you hands-on experience on building a binary classifier using XLNet. If you are unfamiliar with XLNet, or you need to revise it, I strongly recommend you to give this paper a read. If you are unfamiliar with tranformers, I would suggest you to read this paper, or check out this excellent article by Jay Alammar. You don’t necessarily need to know everything about XLNet or Transformers to follow this article, but the above links would help you if you wish to study it further.
The XLNet model was proposed by researchers in Carnegie Mellon University, and Google AI Brain Team. XLNet is an extension of the Transformer-XL model pre-trained using an autoregressive method to learn bidirectional contexts by maximizing the expected likelihood over all permutations of the input sequence factorization order.
XLNet leverages the advantages of both, Auto-regressive and**_ Auto-encoding _**methods for its pretraining which helps it to overcome pretrain-finetune discrepancy.
XLNet can be used for any specific task easily, by downloading the pretrained model and fine-tuning it for the downstream task. To make our work more easy, Huggingface Transformers have already provided few model classes for performing specific downstream tasks using XLNet. We just need to download and fine-tune them, without writing custom model class i.e additional layers on top of the XLNet model.
About Sentiment Classification
Sentiment Classification is a type of Text Classification problem in NLP. We would be performing Binary text classification. The task is to classify whether a given text/document is portraying positive sentiment or negative sentiment.
By classifying the text based on sentiment, can help us get better insights about the entity, for which the feelings have been expressed.
Example of Sentiment classification
In this article, I will walk you through the process of building a binary classifier using XLNet for the IMDB dataset. Code for this article is written in PyTorch. We will use XLNetForSequenceClassification model from Huggingface transformers library to classify the movie reviews.
Let’s dig into what are we going to do!
- Install and import all the dependencies required to set the code working.
- Prepare data
- Writing function to perform train step and evaluation.
- Fine-tuning XLNet model.
- Evaluate performance of the model.
- Making predictions on raw text.
Before we begin, entire code used in this article is available in my github repo.
Let’s start by installing and importing all the dependencies
As a saying goes “Tell me and I forget, teach me and I may remember, involve me and I learn”, it would be better to get your hands on code, so that we get a better idea and understanding of the task.
I have trained the model using google colab, and would recommend the same if you don’t have high end computers. We need to download the transformers library, and PyTorch as we are going to write code using them.
#sentiment-analysis #binary-classification #hugging-face #nlp #xlnet #data analysis
