How to build a transformer model for sentiment analysis (language classification) using HuggingFace’s Transformers library in TensorFlow 2 with Python.

We cover the full process from downloading data all the way through to building and training the transformer model.

Transformers are, without a doubt, one of the biggest advances in NLP in the past decade. They have (quite fittingly) transformed the landscape of language-based ML.Despite this, there are no built-in implementations of transformer models in the core TensorFlow or PyTorch frameworks. To use them, you either need to apply for the relevant Ph.D. program, and we’ll see you in three years — or you pip install transformers.Although this is simplifying the process a little — in reality, it really is incredibly easy to get up and running with some of the most cutting-edge models out there (think BERT and GPT-2).When using Huggingface’s transformers library, we have the option of implementing it via TensorFlow or PyTorch. We will be covering everything you need to know to get started with the TensorFlow flavor in this article.

What is HuggingFace?
  - Finding Models
  - Visualing Attention

TensorFlow
  1\. Pre-Processing
  2\. Tokenizer and Model
  3\. Encoding Inputs
  4\. Full Model Architecture
  5\. Metrics, Loss, and Optimizer
  6\. Training
Results

Full article: https://towardsdatascience.com/tensorflow-and-transformers-df6fceaf57cc

#data-science #python #machine-learning #programming #tensorflow