For a machine learning model to be effective, it must strike a balance between bias and variance. Ridge Regression Regularization aims to decrease the variance, by compromising the bias.

Bias and Variance:

To understand the use of Ridge regression Regularization and its implementation, some context is needed for the problem that it tries to solve.

Bias is essentially the mean squared error of the model, when applied to data that it has been trained on. A model that has a low mean squared error therefore has a low bias.

Variance is a measure of the variation of loss, when comparing the data to different samples from the dataset. Models that have been overfitted have high variance, as their predictions would not function for other samples.

This model has a high bias, as it perfectly fits every point.

However, when fitted against a different sample, in which it has not been trained on, it performs poorly.

Every machine learning algorithm has a trade-off between bias and variance. A general rule of thumb is that a model with more parameters would have a low bias but a larger variance, while a model with less parameters would have a larger bias but a smaller variance.

How does Ridge Regularization decrease the Variance?

As opposed to regression models that minimizes the mean-squared-error function, ridge regularization minimizes the mean-squared-error function added to the product of lambda, and the square of the slope.

#data-science #machine-learning #programming