Introduction

The Data to text generation capability of NLG models is something that I have been exploring since the inception of sequence to sequence models in the field of NLP. The earlier attempts to tackle this problem were not showing any promising results. The non- ML Rule-based approaches like simple NLG did not seem to scale well as they require a well-formatted input and can only perform tasks such as changing the tense of the sentence. But in the age of language models, where new variants of transformers are getting released every two weeks, a task like this is not a far-fetched dream anymore.

In this blog, I will discuss how I approached the Data-to-text generation problem with advanced deep learning models.

The openAI GPT-2 seemed like a good option as it had compelling text generation capabilities. But training it on the web NLG 2017 data didn’t get me anywhere. The model didn’t converge at all. The conditional, as well as the unconditional text generation capabilities of GPT-2, are reasonably good, but you would hardly find a business use case that can be addressed with these tasks.

Furthermore, finetuning them on the domain-specific data at times resulted in the generation of the sentences which were out of context

#naturallanguagegeneration #machine-learning #deep-learning #nlp