The final goal of every deep learning project is to bring value to the product. Of course, we want to have the best possible model. What is “best” — depends on the particular use case, and I will leave this discussion aside from this post. I want to talk about how to get the maximum from your train.py script.

In this post we’re going to cover the following tips:

1. High-level frameworks instead own-made train loops
2. Monitor progress of the training with additional metrics
3. Use TensorBoard
4. Visualize predictions of the model
5. Use Dict as return value for Dataset and Model
6. Detect anomalies and address numerical instabilities

Disclaimer: In the next section, I will include a few source-code listings. Most of them are tailored for the Catalyst framework (version 20.08) and available in pytorch-toolbelt.

Don’t re-invent the wheel

#tips-and-tricks #deep-learning #pytorch #catalyst #deep learning