The most important part of any Machine Learning Model is to know how good or accurate your model is. Okay, so I am a budding Data Scientist and I start building models. But, how do I know the model that I built is good enough. You need to have certain parameters that will define the quality of the model. Isn’t it? Evaluating the quality of the model is very important in improving the model until it performs the best.
So, when it comes to Classification models, the evaluation metrics compare both expected and predicted output to come up with the probability for the class labels. Let’s just understand what is Classification problem. These are the ones where you can clearly see the target variable is divided into classes. If it can be divided into two classes, then it is called Binary Classification Problem and if you can divide it into more than 2 classes then it is called Multi-Class Classification Problem.
So, moving ahead with Evaluation Metrics for Classification Models. Below listed are the metrics used and we will discuss one by one.
1- Accuracy (Not in Case of Imbalanced Classes).
2- Confusion Matrix.
3- Precision.
4- Recall.
5- F1 Score.
6- AUC/ROC.
Let us understand further.
Accuracy:
Okay, let us get this straight into our minds. By Accuracy, what we mean is Classification Accuracy. So, it can be defined as the ration of ‘Number of Correct Prediction’ to Total prediction/Total number of input samples.
Let’s just say we had 5 input out of which we predicted 4 to be correct. Then,
Accuracy = 4/5 = 0.8 = 80%.
Accuracy is one of the simplest Metrics to be used. But, is it the best metrics? Well, the answer is a big NO. Let’s find out why with an example.
Let’s assume that we building a model to predict whether the transaction is fraudulent or not. Well, we built a model with an accuracy 99%. Why is accuracy such high, well it’s because of the class imbalance. Most of the transactions would not be fraudulent. So, if you fit a model that predicts the transaction to be not fraudulent, the accuracy remains 99% owing to class imbalance. Because of the class imbalance, the accuracy shoots up and is not the correct metrics to be used.
#evaluation-metric #data-science #classification-models #machine-learning #confusion-matrix
