How to Reduce Training Time for a Deep Learning Model using tf.data. Learn to create an input pipeline for images to efficiently use CPU and GPU resources to process the image dataset and reduce the training time for a deep learning model.
In this post, you will learn
How does a naive approach work for input data pipeline and model training?
When creating an input data pipeline, typically, we perform the ETL(Extract, Transform, and Load) process.
The pre-processing of the data occurs on the CPU, and the model will be typically trained on GPU/TPU.
In a naive model training approach, CPU pre-processes the data to get it ready for the model to train, while the GPU/TPU is idle. When GPU/TPU starts training the model, the CPU is idle. This is not an efficient way to manage resources as shown below.
Naive data pre-processing and training approach
What are the options to expedite the training process?
To expedite the training, we need to optimize the data extraction, data transformation, and data loading process, all of which happens on the CPU.
Data Extraction: Optimize the data read from data sources
Data Transformation: Parallelize the data augmentation
Data Loading: Prefetch the data one step ahead of training
These techniques will efficiently utilize the CPU and GPU/TPU resources for data pre-processing and training.
How can we achieve the input pipeline optimization?
Experimental evaluation of how the size of the training dataset affects the performance of a classifier trained through Transfer Learning.
In this article, we explore gradient descent - the grandfather of all optimization techniques and it’s variations. We implement them from scratch with Python.
During my studies at JKU there was a task for preprocessing images for a machine learning project. It is necessary to clean the raw images…
Changing Image Backgrounds Using Image Segmentation. Make the world your green screen. Hello readers! In this article I’ll be discussing another cool trick we can do with deep learning.
Burnt out on learning intricate machine learning concepts and complicated jargon? Reignite your passion by building a simple image classifier to detect pneumonia in an x-ray!