Data Generators is one of the most useful features of the Keras API. Consider a scenario where you have lots of data, so much that you cannot have all of it at once in the RAM. Wyd? Purchasing more RAM is obviously isn’t an option.

Well, the solution to this can be loading the mini-batches fed to the model dynamically. This is exactly what data generators do. They can generate the model input dynamically thus forming a pipeline from the storage to the RAM to load the data as and when it is required. Another advantage of this pipeline is, one can easily apply preprocessing routines on these mini-batches of data as they are prepared to feed the model.

In this article, we will see how to subclass the tf.keras.utils.Sequence class to implement custom data generators.

#deep-learning #keras #machine-learning #tensorflow #artificial-intelligence