Inthe previous post, I showed some animated plots for the training process of linear regression and logistic regression. Developing a good “feel” of how they “learn” is helpful because they can be used as a baseline before applying more complex models. Although most deep neural networks also use gradient-based learning, similar intuition is much harder to come by. One reason is that the parameters are very high dimensional and there are a lot of nonlinearities involved, it’s very hard to picture in our heads what is going on during the optimization. Unlike computers, we are only programmed to perceive spaces less or equal to 3D.

#optimization #mathematics #deep-learning #machine-learning #visualization