Where Binary Classification distinguish between two classes, Multiclass Classification or Multinomial Classification can distinguish between more than two classes.

Some algorithms such as SGD classifiers, Random Forest Classifiers, and Naive Bayes classification are capable of handling multiple classes natively. Others such as Logistic Regression or Support Vector Machine Classifiers are strictly binary classifiers. However, there are various strategies that you can use to perform multiclass classification with multiple binary classifiers.

Techniques of Multiclass Classification

There are two Techniques of Multiclass Classification, OvO and OvR, let’s go through both these techniques one by one:

OvR Strategy

One way to create a system that can classify the digit imsges into 10 classes (from 0 to 9) is to train 10 binary classifiers, one for each digit ( a 0 – detector, a 1 – detector, and so on). Then when you want to classify an image, you get the decision score from each classifier for that image and you select the class whose classifier outputs the highest score. This is called the one-versus-the-rest (OvR) strategy also known as one-versus-all.

OvO Strategy

Another strategy is to train a binary classifier for every pair of digits: one to distinguish 0s and 1s, another to distinguish 0s and 2s, another for 1s and 2s, and so on. This is called the one-versus-one (OvO) strategy. If there are N classes, you need to train N × (N – 1)/2 classifiers. For the MNIST problem, this means training 45 binary classifiers. When you want to classify an image, you have to run the image through all 45 classifiers and see which class wins the most duels. The main advantage of OvO is that each classifier only needs to be trained on the part of the training set for the two classes that it must distinguish.

#by aman kharwal #data science #machine learning #multiclass classification #python