40 Algorithms Every Programmer Should Know

The SVM algorithm

Now, let's look at SVM. SVM is a classifier that finds an optimal hyperplane that maximizes the margin between two classes. In SVMs, our optimization objective is to maximize the margin. The margin is defined as the distance between the separating hyperplane (the decision boundary) and the training samples that are closest to this hyperplane, called the support vectors. So, let's start with a very basic example with only two dimensions, X1 and X2. We want a line to separate the circles from the crosses. This is shown in the following diagram:

We have drawn two lines and both perfectly separate the crosses from the circles. ...