Breaking Down the Support Vector Machine (SVM) Algorithm

Breaking Down the Support Vector Machine (SVM) Algorithm

So, I was revising machine learning algorithms yesterday; and I observed that SVM is probably a widely used but complicated machine learning algorithm.

So, I was revising machine learning algorithms yesterday; and I observed that SVM is probably a widely used but complicated machine learning algorithm. So, after understanding the intricacies and coding it from scratch; I thought I must write a post to help individuals like me who find it a little bit more complex than its counterparts.

So, SVM basically uses vector space in order to perform the designated tasks. You’d be surprised to know that SVM is actually better than some neural networks when it comes to recognizing handwritten digits and related tasks. Let’s dive into it!

Basic Intuition

Before we start; you should know that SVM can classify only 2 groups at a single time. But hey; it does not mean it’s limited to binary classifier (more emphasis on the word ‘at a time’). So on an intuitive level; support vector machine separates two categories of data in the dataset with the help of a hyperplane. The perpendicular bisectors from the hyperplane to the datapoints denotes the largest possible distance.

SVM Hyperplane (Decision Boundary) and Perpendicular. Image by Author

In short; even though there can be infinite hyperplanes which can separate the data; the optimum choice is the one with largest separable distances (measure with the perpendicular from the datapoint).

So if a datapoint lies on either side of hyperplane; we can easily tell its category by assuming the hyperplane as a separating wall (just an analogy). But, this can be done with the help of regression too, right? Actually yes, but SVM can handle even more complex separations because it uses vector spaces. Not only this, SVM also does an excellent work in determining the decision boundaries. (That is the best position — max length of perpendicular lines).

Now that we have our basics clear; let’s jump into the technicalities of it.

Let’s stimulate our brains a bit more

In simple terms, we can say that a vector is a quantity which has a magnitude as well as direction (elementary physics/math). In SVM; the magnitude is the numerical value (which can be calculated either by taking the mod or square root of sums of x and y coordinates; similar to hypotenuse in pythagoras theorum).

Apart from that; another thing to know is the dot product of two vectors, which is simply the multiplication of values in two vectors. It can be understood by the figure attached below.

For example if we have a datapoint ‘u’ (which can be represented with u vector) in a dataset which is seperated by a hyperplane (represented by the red line). Now, we take another vector (w) which is drawn perpendicularly from the hyperplane to the origin. Now the dot product of u and w added with a bias decides the output of our support vector assertion.

Now, if the result of our equation is more than or equal to zero; then we can classify the datapoint on the other side (+ve). In the above example; we can visually see that it is not the case (our datapoint is clearly NOT on the +ve side). But even in complex situations the equation works like magic.

Also, you should note that if u.w+b = 0; then the point comes on the decision boundary (a rare situation).

_“But Satyam, this isn’t complicated. That’s easy to understand.” _Well, I’d love if that was the case. But SVM comes with a few constraints on getting the values of bias and vectors. But yes, once you get past this, I’m sure you will love the SVM.

artificial-intelligence machine-learning algorithm

What is Geek Coin

What is GeekCash, Geek Token

Best Visual Studio Code Themes of 2021

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Start a Career in Machine Learning and Artificial Intelligence

Enroll now at best Artificial Intelligence training in Noida, - the best Institute in India for Artificial Intelligence Online Training Course and Certification.

How are deep learning, artificial intelligence and machine learning related

What is the difference between machine learning and artificial intelligence and deep learning? Supervised learning is best for classification and regressions Machine Learning models. You can read more about them in this article.

AI(Artificial Intelligence): The Business Benefits of Machine Learning

Enroll now at CETPA, the best Institute in India for Artificial Intelligence Online Training Course and Certification for students & working professionals & avail 50% instant discount.

Hire Machine Learning Engineer | Offshore Machine Learning Experts

We are a Machine Learning Services provider offering custom AI solutions, Machine Learning as a service & deep learning solutions. Hire Machine Learning experts & build AI Chatbots, Neural networks, etc. 16+ yrs & 2500+ clients.

Learning in Artificial Intelligence - Great Learning

What is Artificial Intelligence (AI)? AI is the ability of a machine to think like human, learn and perform tasks like a human. Know the future of AI, Examples of AI and who provides the course of Artificial Intelligence?