Introduction

This post discusses why logistic regression necessarily uses a different loss function than linear regression. First, the simple yet inefficient way to solve logistic regression will be presented, then the slightly less simple but much more efficient way will be explained and compared.

The simple way

Linear regression is the predecessor of logistic regression for most people studying statistics or machine learning. Some reasons for this might include the following: the equation for making predictions looks just like the y=mx+b equation from high school algebra; the mean squared error cost function can be visualized; it comes with a nice closed-form equation for solving; and best of all, you actually don’t need to know linear algebra or calculus to find the solution.

#log-loss #machine-learning #logistic-regression #cross-entropy #statistics