I had many unsleep nights to get the point how most of the popular Deep Learning Optimization Algorithms are working, how to compare those ones and what is the intuition behind them avoiding very complex math notes.
This is how this video had born.
In this video I analyze all mentioned techniques one after one starting from Momentum, then Rmsprop and finally introduce Adam that is the combination of SGD with Momentum and Rmsprop, as theory says.
Some definitions what is what:
-
Stochatic Gradient Descent with Momentum : 0:55
-
rmsprop (Root Mean Square Propagation) : 2:31
-
Adam (Adaptive moment estimation) : 5:31
-
Comparison of SGD and rmsprop and Adam: 7:09
-
History of Deep Learning Optimization Algorithms: 13:04
I hope that this video will be useful for Deep Learning specialists, Machine Learning Engineers, Data Scientists, Data Analysts who want to learn more than average enthusiast.
Subscribe: https://www.youtube.com/c/VytautasBielinskas/featured
#deep-learning
