In this tutorial, we'll r Learn to use Huggingface and GPT-2 to train a language model to be used with Tensorflow
So it’s been a while since my last article, apologies for that. Work and then the pandemic threw a wrench in a lot of things so I thought I would come back with a little tutorial on text generation with GPT-2 using the Huggingface framework. This will be a Tensorflow focused tutorial since most I have found on google tend to be Pytorch focused, or light on details around using it with Tensorflow. If you don’t want to read my whole post and just see how it works, I have the following Colab notebook as an outline for people to reference here. This post will be basically going over whats in the notebook so should be easy to reference back and forth.
In my last tutorial, I used Markov chains to learn n-gram probabilities from presidential speeches and used those probabilities to generate similar text output given new starting input. Now we will go a step further and utilize a more state of the art architecture to create text output that should be more accurate and realistic. If you haven't already heard about GPT-2, its a language model from OpenAI trained on a mass amount of data from the web using an architecture called the Transformer. Here is a good visual overview of the transformer architecture used by GPT-2 that should help give you intuition on how it works. GPT-2 is not the most advanced version of the language model from Open AI, but its one that has many reference implementations and frameworks to use compared to the newer GPT-3 model. As well its a version of the model that can run on Colab and is fairly straight forward to setup and hopefully even easier after this tutorial :)
In this post, we'll learn DEEP LEARNING WITH TENSORFLOW 2.0 Machine Learning framework.
TensorFlow is the Machine Learning framework by Google. It is primarily used for Deep Learning related tasks and seamlessly integrates with other Google APIs as well. And there are many things about TENSORFLOW 2.0 that you still do not know.
Inexture's Deep learning Development Services helps companies to develop Data driven products and solutions. Hire our deep learning developers today to build application that learn and adapt with time.
In this video I am discussing various techniques to handle imbalanced dataset in machine learning. I also have a python code that demonstrates these different techniques. In the end there is an exercise for you to solve along with a solution link. Credit card fraud detection, cancer prediction, customer churn prediction are some of the examples where you might get an imbalanced dataset. Training a model on imbalanced dataset requires making certain adjustments otherwise the model will not perform as per your expectations.
Can intelligence emerge simply by training a big enough language model using lots of data? OpenAI tries to do so, using 175 billion parameters.