I am writing a new series of (relatively short) posts centered around foundational topics in statistical learning. In particular, this series will feature unexpected discoveries, less-talked-about linkages, and under-the-hood concepts for statistical learning.

My first post starts with ridge regularization, an essential concept in Data Science. Simple yet elegant relationships between ordinary least squares (OLS) estimates, ridge estimates, and PCA can be found through the lens of spectral decomposition. We see these relationships through Exercise 8.8.1 of Multivariate Analysis.

This article is adapted from one of my blog posts with all the proofs omitted. If you prefer LaTex-formatted maths and HTML style pages, you can read this article on my blog.

#ridge-regression #machine-learning #data-science #supervised-learning