Applied Machine Learning and AI for Engineers

Chapter 5. Support Vector Machines

Support vector machines (SVMs) represent the cutting edge of machine learning. They are most often used to solve classification problems, but they can also be used for regression. Due to the unique way in which they fit mathematical models to data, SVMs often succeed at finding separation between classes when other models do not. They technically perform binary classification only, but Scikit-Learn enables them to do multiclass classification as well using techniques discussed in Chapter 3.

Scikit-Learn makes building SVMs easy with classes such as SVC (short for support vector classifier) for classification models and SVR (support vector regressor) for regression models. You can use these classes without understanding how SVMs work, but you’ll get more out of them if you do understand how they work. It’s also important to know how to tune SVMs for individual datasets and how to prepare data before you train a model. Toward the end of this chapter, we’ll build an SVM that performs facial recognition. But first, let’s look behind the scenes and discover why SVMs are often the go-to mechanism for modeling real-world datasets.

How Support Vector Machines Work

First, why are they called support vector machines? The purpose of an SVM classifier is the same as any other classifier: to find a decision boundary that cleanly separates the classes. SVMs do this by finding a line in 2D space, a plane in 3D space, or a hyperplane in higher-dimensional space ...