book

Learning Apache Mahout

Name: Learning Apache Mahout
Author: Chandramani Tiwary
ISBN: 9781783555215

by Chandramani Tiwary

March 2015

Beginner to intermediate

250 pages

5h 39m

English

Packt Publishing

Read now

Unlock full access

Learning Apache Mahout
Table of Contents
Learning Apache Mahout
Credits
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers, and moreWhy subscribe?Free access for Packt account holders
Preface
What this book covers
What you need for this book
Who this book is for

Conventions
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions
1. Introduction to Mahout
Why MahoutSimple techniques and more data is betterSampling is difficultCommunity and license
When Mahout
Data too large for single machineData already on HadoopAlgorithms implemented in Mahout
How Mahout
Setting up the development environmentConfiguring MavenConfiguring MahoutConfiguring Eclipse with the Maven plugin and MahoutMahout command lineA clustering exampleReuter's raw data fileA classification exampleMahout API – a Java program exampleThe datasetParallel versus in-memory execution mode
Summary
2. Core Concepts in Machine Learning
Supervised learningDetermine the objectiveDecide the training dataCreate and clean the training setFeature extractionTrain the modelsBaggingBoostingValidationHoldout-set validationK-fold cross validationEvaluationBias-variance trade-offFunction complexity and amount of training dataDimensionality of the input spaceNoise in data
Unsupervised learning
Cluster analysisObjectiveFeature representationFeature normalizationRow normalizationColumn normalizationRescalingStandardizationA notion of similarity and dissimilarityEuclidean distance measureSquared Euclidean distance measureManhattan distance measureTanimoto distance measureAlgorithm for clusteringA stopping criteriaFrequent pattern miningMeasures for identifying interesting rulesSupportConfidenceLiftConvictionThings to considerActionable rulesWhat association to look for
Recommender system
Collaborative filteringCold startScalabilitySparsityContent-based filtering
Model efficacy
ClassificationConfusion matrixROC curve and AUCFeatures of ROC graphsEvaluating classifier using the ROC curveArea-based accuracy measureEuclidian distance comparisonRegressionMean absolute errorRoot mean squared errorR-squareAdjusted R-squareRecommendation systemScore differencePrecision and recallClusteringThe internal evaluationThe intra-cluster distanceThe inter-cluster distanceThe Davies–Bouldin indexThe Dunn indexThe external evaluationThe Rand indexF-measure
Summary
3. Feature Engineering
Feature engineeringFeature constructionCategorical featuresMerging categoriesConverting to binary variablesConverting to continuous variablesContinuous featuresBinningBinarizationFeature standardizationRescalingMean standardizationScaling to unit normFeature transformation derived from the problem domainRatiosFrequencyAggregate transformationsNormalizationMathematical transformationsFeature extractionFeature selectionFilter-based feature selectionWrapper-based feature selectionBackward selectionForward selectionRecursive feature eliminationEmbedded feature selectionDimensionality reduction
Summary
4. Classification with Mahout
ClassificationWhite box modelsBlack box models
Logistic regression
Mahout logistic regression command lineGetting the dataModel building via command lineSplitting the datasetTrain the model command line optionInterpreting the outputTesting the modelPrediction
Adaptive regression model
Code example with logistic regression
Train the modelThe LogisticRegressionParameter and CsvRecordFactory classesA code example without the parameter classTesting the online regression modelGetting predictions from OnlineLogisticRegressionA CrossFoldLearner example
Random forest
BaggingRandom subsets of featuresOut-of-bag error estimateRandom forest using the command linePredictions from random forest
Naïve Bayes classifier
Numeric features with naïve BayesCommand line
Summary
5. Frequent Pattern Mining and Topic Modeling
Frequent pattern miningBuilding FP TreeConstructing the treeIdentifying frequent patterns from FP Tree
Importing the Mahout source code into Eclipse
Frequent pattern mining with Mahout
Extending the command line of MahoutGetting the dataData descriptionFrequent pattern mining with Mahout APIMapReduce executionLinear executionFormatting the results and computing metricsTopic modeling using LDALDA using the Mahout command line
Summary
6. Recommendation with Mahout
Collaborative filteringSimilarity measuresPearson correlation similarityEuclidean distance similarityComputing similarity without a preference valueTanimoto coefficient similarityLog-likelihood similarityEvaluating recommenderUser-based recommender systemUser neighborhoodFixed size neighborhoodThreshold-based neighborhoodThe datasetMahout code exampleBuilding the recommenderEvaluating the recommenderItem-based recommender systemMahout code exampleBuilding the recommenderEvaluating the recommenderInferring preferences
Summary
7. Clustering with Mahout
k-meansDeciding the number of clustersDeciding the initial centroidRandom pointsPoints from the datasetPartition by rangeCanopy centroidsAdvantages and disadvantages
Canopy clustering
Fuzzy k-means
Deciding the fuzzy factor
A Mahout command-line example
Getting the dataPreprocessing the datak-meansCanopy clusteringFuzzy k-meansStreaming k-means
A Mahout Java example
k-meansCluster evaluation
Summary
8. New Paradigm in Mahout
Moving beyond MapReduce
Apache Spark
Configuring Spark with MahoutBasics of Mahout Scala DSLImports
In-core types
VectorInitializing a vector inlineAccessing elements of a vectorSetting values of an elementVector arithmeticVector operations with a scalarMatrixInitializing the matrixAccessing elements of a matrixSetting the matrix columnCopy by reference
Spark Mahout basics
Initializing the Spark contextOptimizer actionsComputational actionsCaching in Spark's block manager
Linear regression with Mahout Spark
Summary
9. Case Study – Churn Analytics and Customer Segmentation
Churn analyticsGetting the dataData explorationInstalling RSummary statisticsCorrelationFeature engineeringModel training and validationLogistic regressionAdaptive logistic regressionRandom forestCustomer segmentationPreprocessingFeature extractionDay callsEvening callsInternational callsPreprocessing the filesCreating the clusters using fuzzy k-meansClustering using k-meansEvaluation
Summary
10. Case Study – Text Analytics
Text analyticsVector space modelPreprocessingTokenizationStop word removalStemmingPreprocessing exampleDocument indexingTF-IDF weightingn-gramsNormalization
Clustering text
The datasetFeature extractionThe clustering job
Categorizing text
The datasetFeature extractionThe classification job
Summary
Index

Content preview from Learning Apache Mahout

Summary

Machine learning adoption has increased in leaps and bounds in the last few years. Availability of data in the digital age, scalable platforms to process large amount of data, focus on intelligent applications, and the realization of the business values of machine learning has driven the growth. Machine learning is a vast field that requires continuing education to gain expertise. This chapter has introduced a few important concepts in machine learning but the knowledge gained is just the tip of the iceberg. I would encourage the reader to keep exploring and augmenting their knowledge. In the next chapter, we will discuss one of the first and most important parts of a machine learning project, feature engineering. Feature engineering deals ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781783555215

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Learning Apache Mahout

by Chandramani Tiwary

Summary

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.