Introduction

In machine learning, usually, there is a loss function (or cost function) that we need to find the minimal value. Gradient Descent is one of the optimization methods that is widely applied to do the job.

In this post, I will discuss the gradient descent method with some examples including linear regression using PyTorch.

Calculus

One method to find a function’s max or min, it to find the point(s) where the slope equals zero. The max or min of the function will be the solution of the derivative of a function equals zero.

#machine-learning #programming #data-science #gradient-descent #deep-learning