In this article, we will try to understand the Model Sub-Classing API and Custom Training Loop from Scratch in TensorFlow 2. It may not be a beginner or advance introduction but aim to get rough intuition of what they are all about. The post is divided into three parts:
- Comparable Modelling Strategies in TensorFlow 2
- Build an Inception Network with Model Sub-Classing API
- End-to-End Training with Custom Training Loop from Scratch
So, at first, we will see how many ways to define models using TensorFlow 2 and how they differ from each other. Next, we will see how feasible it is to build a complex neural architecture using the model subclassing API which is introduced in TF 2. And then we will implement a custom training loop and train these subclassing model end-to-end from scratch. We will also use Tensorboard in our custom training loop to track the model performance for each batch. We will also see how to save and load the model after training. In the end, we will measure the model performance via the confusion matrix and classification report, etc.
Comparable Modelling Strategies in TensorFlow 2
In TF.Keras there are basically three-way we can define a neural network, namely
- Sequential API
- Functional API
- Model Subclassing API
Among them, Sequential API is the easiest way to implement but comes with certain limitations. For example, with this API, we can’t create a model that shares feature information to another layer except to its subsequent layer. In addition, multiple input and output are not possible to implement either. In this point, Functional API does solve these issues greatly. A model like Inception or ResNet is feasible to implement in Functional API. But often deep learning researcher wants to have more control over every nuance of the network and on the training pipelines and that’s exactly what Model Subclassing API serves. Model Sub-Classing is a fully customizable way to implement the feed-forward mechanism for our custom-designed deep neural network in an object-oriented fashion.
#deep-learning #machine-learning #research #tensorflow #keras
