book

Applied Unsupervised Learning with R

Name: Applied Unsupervised Learning with R
ISBN: 9781789956399

by Alok Malik, Bradford Tuckfield

March 2019

Intermediate to advanced

320 pages

5h 28m

English

Packt Publishing

Read now

Unlock full access

Preface
About the BookAbout the AuthorsElevator PitchKey FeaturesDescriptionLearning ObjectivesAudienceApproachHardware RequirementsSoftware RequirementsConventionsInstallation and SetupInstalling R on WindowsInstalling R on macOS XInstalling R on Linux
Chapter 1
Introduction to Clustering MethodsIntroductionIntroduction to Clustering Uses of ClusteringIntroduction to the Iris DatasetExercise 1: Exploring the Iris DatasetTypes of ClusteringIntroduction to k-means ClusteringEuclidean DistanceManhattan DistanceCosine DistanceThe Hamming Distancek-means Clustering AlgorithmSteps to Implement k-means ClusteringExercise 2: Implementing k-means Clustering on the Iris DatasetActivity 1: k-means Clustering with Three ClustersIntroduction to k-means Clustering with Built-In Functionsk-means Clustering with Three ClustersExercise 3: k-means Clustering with R LibrariesIntroduction to Market SegmentationExercise 4: Exploring the Wholesale Customer DatasetActivity 2: Customer Segmentation with k-means Introduction to k-medoids ClusteringThe k-medoids Clustering Algorithmk-medoids Clustering CodeExercise 5: Implementing k-medoid Clusteringk-means Clustering versus k-medoids ClusteringActivity 3: Performing Customer Segmentation with k-medoids ClusteringDeciding the Optimal Number of ClustersTypes of Clustering MetricsSilhouette ScoreExercise 6: Calculating the Silhouette ScoreExercise 7: Identifying the Optimum Number of ClustersWSS/Elbow MethodExercise 8: Using WSS to Determine the Number of ClustersThe Gap StatisticExercise 9: Calculating the Ideal Number of Clusters with the Gap StatisticActivity 4: Finding the Ideal Number of Market SegmentsSummary
Chapter 2
Advanced Clustering MethodsIntroductionIntroduction to k-modes ClusteringSteps for k-Modes ClusteringExercise 10: Implementing k-modes ClusteringActivity 5: Implementing k-modes Clustering on the Mushroom DatasetIntroduction to Density-Based Clustering (DBSCAN)Steps for DBSCAN Exercise 11: Implementing DBSCAN Uses of DBSCAN Activity 6: Implementing DBSCAN and Visualizing the ResultsIntroduction to Hierarchical ClusteringTypes of Similarity MetricsSteps to Perform Agglomerative Hierarchical ClusteringExercise 12: Agglomerative Clustering with Different Similarity MeasuresDivisive ClusteringSteps to Perform Divisive ClusteringExercise 13: Performing DIANA Clustering Activity 7: Performing Hierarchical Cluster Analysis on the Seeds DatasetSummary
Chapter 3
Probability DistributionsIntroductionBasic Terminology of Probability DistributionsUniform DistributionExercise 14: Generating and Plotting Uniform Samples in RNormal DistributionExercise 15: Generating and Plotting a Normal Distribution in RSkew and KurtosisLog-Normal DistributionsExercise 16: Generating a Log-Normal Distribution from a Normal DistributionThe Binomial DistributionExercise 17: Generating a Binomial DistributionThe Poisson DistributionThe Pareto DistributionIntroduction to Kernel Density EstimationKDE AlgorithmExercise 18: Visualizing and Understanding KDEExercise 19: Studying the Effect of Changing Kernels on a DistributionActivity 8: Finding the Standard Distribution Closest to the Distribution of Variables of the Iris DatasetIntroduction to the Kolmogorov-Smirnov TestThe Kolmogorov-Smirnov Test AlgorithmExercise 20: Performing the Kolmogorov-Smirnov Test on Two SamplesActivity 9: Calculating the CDF and Performing the Kolmogorov-Smirnov Test with the Normal DistributionSummary
Chapter 4
Dimension ReductionIntroductionThe Idea of Dimension ReductionExercise 21: Examining a Dataset that Contains the Chemical Attributes of Different WinesImportance of Dimension ReductionMarket Basket AnalysisExercise 22: Data Preparation for the Apriori AlgorithmExercise 23: Passing through the Data to Find the Most Common BasketsExercise 24: More Passes through the DataExercise 25: Generating Associative Rules as the Final Step of the Apriori AlgorithmPrincipal Component AnalysisLinear Algebra RefresherMatricesVarianceCovarianceExercise 26: Examining Variance and Covariance on the Wine DatasetEigenvectors and EigenvaluesThe Idea of PCAExercise 27: Performing PCAExercise 28: Performing Dimension Reduction with PCAActivity 10: Performing PCA and Market Basket Analysis on a New DatasetSummary
Chapter 5
Data Comparison MethodsIntroductionHash FunctionsExercise 29: Creating and Using a Hash FunctionExercise 30: Verifying Our Hash FunctionAnalytic SignaturesExercise 31: Perform the Data Preparation for Creating an Analytic Signature for an Image Exercise 32: Creating a Brightness Comparison FunctionExercise 33: Creating a Function to Compare Image Sections to All of the Neighboring SectionsExercise 34: Creating a Function that Generates an Analytic Signature for an ImageActivity 11: Creating an Image Signature for a Photograph of a PersonComparison of SignaturesActivity 12: Creating an Image Signature for the Watermarked ImageApplying Other Unsupervised Learning Methods to Analytic Signatures Latent Variable Models – Factor AnalysisExercise 35: Preparing for Factor AnalysisLinear Algebra behind Factor AnalysisExercise 36: More Exploration with Factor Analysis Activity 13: Performing Factor AnalysisSummary
Chapter 6
Anomaly DetectionIntroductionUnivariate Outlier DetectionExercise 37: Performing an Exploratory Visual Check for Outliers Using R's boxplot FunctionExercise 38: Transforming a Fat-Tailed Dataset to Improve Outlier ClassificationExercise 39: Finding Outliers without Using R's Built-In boxplot FunctionExercise 40: Detecting Outliers Using a Parametric MethodMultivariate Outlier DetectionExercise 41: Calculating Mahalanobis DistanceDetecting Anomalies in ClustersOther Methods for Multivariate Outlier DetectionExercise 42: Classifying Outliers based on Comparisons of Mahalanobis DistancesDetecting Outliers in Seasonal DataExercise 43: Performing Seasonality ModelingExercise 44: Finding Anomalies in Seasonal Data Using a Parametric MethodContextual and Collective AnomaliesExercise 45: Detecting Contextual AnomaliesExercise 46: Detecting Collective AnomaliesKernel DensityExercise 47: Finding Anomalies Using Kernel Density EstimationContinuing in Your Studies of Anomaly DetectionActivity 14: Finding Univariate Anomalies Using a Parametric Method and a Non-parametric MethodActivity 15: Using Mahalanobis Distance to Find AnomaliesSummary
Appendix
Chapter 1: Introduction to Clustering MethodsActivity 1: k-means Clustering with Three ClustersActivity 2: Customer Segmentation with k-means Activity 3: Performing Customer Segmentation with k-medoids Clustering Activity 4: Finding the Ideal Number of Market SegmentsChapter 2: Advanced Clustering MethodsActivity 5: Implementing k-modes Clustering on the Mushroom DatasetActivity 6: Implementing DBSCAN and Visualizing the ResultsActivity 7: Performing a Hierarchical Cluster Analysis on the Seeds DatasetChapter 3: Probability DistributionsActivity 8: Finding the Standard Distribution Closest to the Distribution of Variables of the Iris DatasetActivity 9: Calculating the CDF and Performing the Kolmogorov-Simonov Test with the Normal DistributionChapter 4: Dimension ReductionActivity 10: Performing PCA and Market Basket Analysis on a New DatasetChapter 5: Data Comparison MethodsActivity 11: Create an Image Signature for a Photograph of a PersonActivity 12: Create an Image Signature for the Watermarked ImageActivity 13: Performing Factor AnalysisChapter 6: Anomaly DetectionActivity 14: Finding Univariate Anomalies Using a Parametric Method and a Non-parametric MethodActivity 15: Using Mahalanobis Distance to Find Anomalies

Content preview from Applied Unsupervised Learning with R

Chapter 6 Anomaly Detection

Learning Objectives

By the end of this chapter, you will be able to:

Use parametric and non-parametric methods to find outliers in univariate and multivariate data
Use data transformations to identify outliers in univariate and multivariate data
Work with Mahalanobis distances
Improve anomaly detection performance by incorporating a model of seasonality

In this chapter, we will have a look at different anomaly detection techniques.

Introduction

Data analysis often begins with an implicit assumption that all observations are valid, accurate, and trustworthy. But this is not always a reasonable assumption. Consider the case of credit card companies, who collect data consisting of records of charges to an individual's ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781789956399

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Applied Unsupervised Learning with R

by Alok Malik, Bradford Tuckfield

Chapter 6

Anomaly Detection

Learning Objectives

Introduction

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.