Introduction

Researchers at Google AI released the paper “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding” 2 years ago. Since then, it had gained a lot of popularity in the Machine learning/ Natural Language Processing world. BERT was built on top of many successful and promising work that has been popular in the NLP world recently. Including, but not limited to, Seq2Seq architectures, Transformer (from the “Attention is All You Need Paper”) ELMO, ULM-FIT and unsupervised language modeling. At the time BERT was released, BERT showed state-of-the-art performances on a wide variety of NLP tasks. In this article, we will be fine-tuning a pre-trained Turkish BERT model on a Turkish Named Entity Recognition (NER) dataset. We will be using the popular HuggingFace pre-trained transformers library for the fine-tuning stage.We will also implement a known solution for handling BERT maximum sequence length problem by building overlapping sub-sequences.

In this article, I’m making the assumption that the readers already have background information on the following subjects:

1. Named Entity Recognition (NER).
2. Bidirectional Encoder Representations from Transformers (BERT).
3. HuggingFace (transformers) Python library.

Focus of this article:

1. Utilize HuggingFace Trainer class to easily fine-tune BERT model for the NER task (applicable to most transformers not just BERT).
2. Handling sequences longer than BERT’s MAX_LEN = 512

HuggingFace Trainer Class:

Transformers new Trainer class provides an easy way of fine-tuning transformer models for known tasks such as CoNLL NER. Here are other supported tasks. This class will take care of training/evaluation loops, logging, model saving …etc. Which makes switching to other transformers models very easy. For this purpose we will use another class NerDataset which handles loading and tokenization of the data.

#deep-learning #hugging-face #nlp #named-entity-recognition #bert