Hands-On Unsupervised Learning Using Python

Chapter 4. Anomaly Detection

In Chapter 3, we introduced the core dimensionality reduction algorithms and explored their ability to capture the most salient information in the MNIST digits database in significantly fewer dimensions than the original 784 dimensions. Even in just two dimensions, the algorithms meaningfully separated the digits, without using labels. This is the power of unsupervised learning algorithms—they can learn the underlying structure of data and help discover hidden patterns in the absence of labels.

Let’s build an applied machine learning solution using these dimensionality reduction methods. We will turn to the problem we introduced in Chapter 2 and build a credit card fraud detection system without using labels.

In the real world, fraud often goes undiscovered, and only the fraud that is caught provides any labels for the datasets. Moreover, fraud patterns change over time, so supervised systems that are built using fraud labels—like the one we built in Chapter 2—become stale, capturing historical patterns of fraud but failing to adapt to newly emerging patterns.

For these reasons (the lack of sufficient labels and the need to adapt to newly emerging patterns of fraud as quickly as possible), unsupervised learning fraud detection systems are in vogue.

In this chapter, we will build such a solution using some of the dimensionality reduction algorithms we explored in the previous chapter.

Credit Card Fraud Detection

Let’s revisit the credit card transactions ...