book

R: Predictive Analysis

Name: R: Predictive Analysis
ISBN: 9781788290371

by Tony Fischetti, Eric Mayor, Rui Miguel Forte

March 2017

Beginner to intermediate

1065 pages

27h 7m

English

Packt Publishing

Read now

Unlock full access

R: Predictive Analysis
Table of Contents
R: Predictive Analysis
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions
1. Module 1

1. RefresheR
Navigating the basicsArithmetic and assignmentLogicals and charactersFlow of control
Getting help in R
Vectors
SubsettingVectorized functionsAdvanced subsettingRecycling
Functions
Matrices
Loading data into R
Working with packages
Exercises
Summary
2. The Shape of Data
Univariate data
Frequency distributions
Central tendency
Spread
Populations, samples, and estimation
Probability distributions
Visualization methods
Exercises
Summary
3. Describing Relationships
Multivariate data
Relationships between a categorical and a continuous variable
Relationships between two categorical variables
The relationship between two continuous variables
CovarianceCorrelation coefficientsComparing multiple correlations
Visualization methods
Categorical and continuous variablesTwo categorical variablesTwo continuous variablesMore than two continuous variables
Exercises
Summary
4. Probability
Basic probability
A tale of two interpretations
Sampling from distributions
ParametersThe binomial distribution
The normal distribution
The three-sigma rule and using z-tables
Exercises
Summary
5. Using Data to Reason About the World
Estimating means
The sampling distribution
Interval estimation
How did we get 1.96?
Smaller samples
Exercises
Summary
6. Testing Hypotheses
Null Hypothesis Significance TestingOne and two-tailed testsWhen things go wrongA warning about significanceA warning about p-values
Testing the mean of one sample
Assumptions of the one sample t-test
Testing two means
Don't be fooled!Assumptions of the independent samples t-test
Testing more than two means
Assumptions of ANOVA
Testing independence of proportions
What if my assumptions are unfounded?
Exercises
Summary
7. Bayesian Methods
The big idea behind Bayesian analysis
Choosing a prior
Who cares about coin flips
Enter MCMC – stage left
Using JAGS and runjags
Fitting distributions the Bayesian way
The Bayesian independent samples t-test
Exercises
Summary
8. Predicting Continuous Variables
Linear models
Simple linear regression
Simple linear regression with a binary predictor
A word of warning
Multiple regression
Regression with a non-binary predictor
Kitchen sink regression
The bias-variance trade-off
Cross-validationStriking a balance
Linear regression diagnostics
Second Anscombe relationshipThird Anscombe relationshipFourth Anscombe relationship
Advanced topics
Exercises
Summary
9. Predicting Categorical Variables
k-Nearest NeighborsUsing k-NN in RConfusion matricesLimitations of k-NN
Logistic regression
Using logistic regression in R
Decision trees
Random forests
Choosing a classifier
The vertical decision boundaryThe diagonal decision boundaryThe crescent decision boundaryThe circular decision boundary
Exercises
Summary
10. Sources of Data
Relational DatabasesWhy didn't we just do that in SQL?
Using JSON
XML
Other data formats
Online repositories
Exercises
Summary
11. Dealing with Messy Data
Analysis with missing dataVisualizing missing dataTypes of missing dataSo which one is it?Unsophisticated methods for dealing with missing dataComplete case analysisPairwise deletionMean substitutionHot deck imputationRegression imputationStochastic regression imputationMultiple imputationSo how does mice come up with the imputed values?Methods of imputationMultiple imputation in practice
Analysis with unsanitized data
Checking for out-of-bounds dataChecking the data type of a columnChecking for unexpected categoriesChecking for outliers, entry errors, or unlikely data pointsChaining assertions
Other messiness
OpenRefineRegular expressionstidyr
Exercises
Summary
12. Dealing with Large Data
Wait to optimize
Using a bigger and faster machine
Be smart about your code
Allocation of memoryVectorization
Using optimized packages
Using another R implementation
Use parallelization
Getting started with parallel RAn example of (some) substance
Using Rcpp
Be smarter about your code
Exercises
Summary
13. Reproducibility and Best Practices
R ScriptingRStudioRunning R scriptsAn example scriptScripting and reproducibility
R projects
Version control
Communicating results
Exercises
Summary
2. Module 2
1. Visualizing and Manipulating Data Using R
The roulette case
Histograms and bar plots
Scatterplots
Boxplots
Line plots
Application – Outlier detection
Formatting plots
Summary
2. Data Visualization with Lattice
Loading and discovering the lattice package
Discovering multipanel conditioning with xyplot()
Discovering other lattice plots
HistogramsStacked barsDotplotsDisplaying data points as text
Updating graphics
Case study – exploring cancer-related deaths in the US
Discovering the datasetIntegrating supplementary external data
Summary
3. Cluster Analysis
Distance measures
Learning by doing – partition clustering with kmeans()
Setting the centroidsComputing distances to centroidsComputing the closest cluster for each caseTasks performed by the main functionInternal validation
Using k-means with public datasets
Understanding the data with the all.us.city.crime.1970 datasetFinding the best number of clusters in the life.expectancy.1971 datasetExternal validation
Summary
4. Agglomerative Clustering Using hclust()
The inner working of agglomerative clustering
Agglomerative clustering with hclust()
Exploring the results of votes in SwitzerlandThe use of hierarchical clustering on binary attributes
Summary
5. Dimensionality Reduction with Principal Component Analysis
The inner working of Principal Component Analysis
Learning PCA in R
Dealing with missing valuesSelecting how many components are relevantNaming the components using the loadingsPCA scoresAccessing the PCA scoresPCA scores for analysisPCA diagnostics
Summary
6. Exploring Association Rules with Apriori
Apriori – basic conceptsAssociation rulesItemsetsSupportConfidenceLift
The inner working of apriori
Generating itemsets with support-based pruningGenerating rules by using confidence-based pruning
Analyzing data with apriori in R
Using apriori for basic analysisDetailed analysis with aprioriPreparing the dataAnalyzing the dataCoercing association rules to a data frameVisualizing association rules
Summary
7. Probability Distributions, Covariance, and Correlation
Probability distributionsIntroducing probability distributionsDiscrete uniform distributionThe normal distributionThe Student's t-distributionThe binomial distributionThe importance of distributions
Covariance and correlation
CovarianceCorrelationPearson's correlationSpearman's correlation
Summary
8. Linear Regression
Understanding simple regressionComputing the intercept and slope coefficientObtaining the residualsComputing the significance of the coefficient
Working with multiple regression
Analyzing data in R: correlation and regression
First steps in the data analysisPerforming the regressionChecking for the normality of residualsChecking for variance inflationExamining potential mediations and comparing modelsPredicting new data
Robust regression
Bootstrapping
Summary
9. Classification with k-Nearest Neighbors and Naïve Bayes
Understanding k-NN
Working with k-NN in R
How to select k
Understanding Naïve Bayes
Working with Naïve Bayes in R
Computing the performance of classification
Summary
10. Classification Trees
Understanding decision trees
ID3
EntropyInformation gain
C4.5
The gain ratioPost-pruning
C5.0
Classification and regression trees and random forest
CARTRandom forestBagging
Conditional inference trees and forests
Installing the packages containing the required functions
Installing C4.5Installing C5.0Installing CARTInstalling random forestInstalling conditional inference treesLoading and preparing the data
Performing the analyses in R
Classification with C4.5The unpruned treeThe pruned treeC50CARTPruningRandom forests in RExamining the predictions on the testing setConditional inference trees in R
Caret – a unified framework for classification
Summary
12. Multilevel Analyses
Nested data
Multilevel regression
Random intercepts and fixed slopesRandom intercepts and random slopes
Multilevel modeling in R
The null modelRandom intercepts and fixed slopesRandom intercepts and random slopes
Predictions using multilevel models
Using the predict() functionAssessing prediction quality
Summary
13. Text Analytics with R
An introduction to text analytics
Loading the corpus
Data preparation
Preprocessing and inspecting the corpusComputing new attributes
Creating the training and testing data frames
Classification of the reviews
Document classification with k-NNDocument classification with Naïve BayesClassification using logistic regressionDocument classification with support vector machines
Mining the news with R
A successful document classificationExtracting the topics of the articlesCollecting news articles in R from the New York Times article search API
Summary
14. Cross-validation and Bootstrapping Using Caret and Exporting Predictive Models Using PMML
Cross-validation and bootstrapping of predictive models using the caret packageCross-validationPerforming cross-validation in R with caretBootstrappingPerforming bootstrapping in R with caretPredicting new data
Exporting models using PMML
What is PMML?A brief description of the structure of PMML objectsExamples of predictive model exportationExporting k-means objectsHierarchical clusteringExporting association rules (apriori objects)Exporting Naïve Bayes objectsExporting decision trees (rpart objects)Exporting random forest objectsExporting logistic regression objectsExporting support vector machine objects
Summary
A. Exercises and Solutions
ExercisesChapter 1 – Setting GNU R for Predictive ModelingChapter 2 – Visualizing and Manipulating Data Using RChapter 3 – Data Visualization with LatticeChapter 4 – Cluster AnalysisChapter 5 – Agglomerative Clustering Using hclust()Chapter 6 – Dimensionality Reduction with Principal Component AnalysisChapter 7 – Exploring Association Rules with AprioriChapter 8 – Probability Distributions, Covariance, and CorrelationChapter 9 – Linear RegressionChapter 10 – Classification with k-Nearest Neighbors and Naïve BayesChapter 11 – Classification TreesChapter 12 – Multilevel AnalysesChapter 13 – Text Analytics with R
Solutions
Chapter 1 – Setting GNU R for Predictive ModelingChapter 2 – Visualizing and Manipulating Data Using RChapter 3 – Data Visualization with LatticeChapter 4 – Cluster AnalysisChapter 5 – Agglomerative Clustering Using hclust()Chapter 6 – Dimensionality Reduction with Principal Component AnalysisChapter 7 – Exploring Association Rules with AprioriChapter 8 – Probability Distributions, Covariance, and CorrelationChapter 9 – Linear RegressionChapter 10 – Classification with k-Nearest Neighbors and Naïve BayesChapter 11 – Classification TreesChapter 12 – Multilevel AnalysesChapter 13 – Text Analytics with R
B. Further Reading and References
Preface
Chapter 1 – Setting GNU R for Predictive Modeling
Chapter 2 – Visualizing and Manipulating Data Using R
Chapter 3 – Data Visualization with Lattice
Chapter 4 – Cluster Analysis
Chapter 5 – Agglomerative Clustering Using hclust()
Chapter 6 – Dimensionality Reduction with Principal Component Analysis
Chapter 7 – Exploring Association Rules with Apriori
Chapter 8 – Probability Distributions, Covariance, and Correlation
Chapter 9 – Linear Regression
Chapter 10 – Classification with k-Nearest Neighbors and Naïve Bayes
Chapter 11 – Classification Trees
Chapter 12 – Multilevel Analyses
Chapter 13 – Text Analytics with R
Chapter 14 – Cross-validation and Bootstrapping Using Caret and Exporting Predictive Models Using PMML
3. Module 3
1. Gearing Up for Predictive Modeling
ModelsLearning from dataThe core components of a modelOur first model: k-nearest neighbors
Types of models
Supervised, unsupervised, semi-supervised, and reinforcement learning modelsParametric and nonparametric modelsRegression and classification modelsReal-time and batch machine learning models
The process of predictive modeling
Defining the model's objectiveCollecting the dataPicking a modelPreprocessing the dataExploratory data analysisFeature transformationsEncoding categorical featuresMissing dataOutliersRemoving problematic featuresFeature engineering and dimensionality reductionTraining and assessing the modelRepeating with different models and final model selectionDeploying the model
Performance metrics
Assessing regression modelsAssessing classification modelsAssessing binary classification models
Summary
2. Linear Regression
Introduction to linear regressionAssumptions of linear regression
Simple linear regression
Estimating the regression coefficients
Multiple linear regression
Predicting CPU performancePredicting the price of used cars
Assessing linear regression models
Residual analysisSignificance tests for linear regressionPerformance metrics for linear regressionComparing different regression modelsTest set performance
Problems with linear regression
MulticollinearityOutliers
Feature selection
Regularization
Ridge regressionLeast absolute shrinkage and selection operator (lasso)Implementing regularization in R
Summary
3. Logistic Regression
Classifying with linear regression
Introduction to logistic regression
Generalized linear modelsInterpreting coefficients in logistic regressionAssumptions of logistic regressionMaximum likelihood estimation
Predicting heart disease
Assessing logistic regression models
Model devianceTest set performance
Regularization with the lasso
Classification metrics
Extensions of the binary logistic classifier
Multinomial logistic regressionPredicting glass typeOrdinal logistic regressionPredicting wine quality
Summary
4. Neural Networks
The biological neuron
The artificial neuron
Stochastic gradient descent
Gradient descent and local minimaThe perceptron algorithmLinear separationThe logistic neuron
Multilayer perceptron networks
Training multilayer perceptron networks
Predicting the energy efficiency of buildings
Evaluating multilayer perceptrons for regression
Predicting glass type revisited
Predicting handwritten digits
Receiver operating characteristic curves
Summary
5. Support Vector Machines
Maximal margin classification
Support vector classification
Inner products
Kernels and support vector machines
Predicting chemical biodegration
Cross-validation
Predicting credit scores
Multiclass classification with support vector machines
Summary
6. Tree-based Methods
The intuition for tree models
Algorithms for training decision trees
Classification and regression treesCART regression treesTree pruningMissing dataRegression model treesCART classification treesC5.0
Predicting class membership on synthetic 2D data
Predicting the authenticity of banknotes
Predicting complex skill learning
Tuning model parameters in CART treesVariable importance in tree modelsRegression model trees in action
Summary
7. Ensemble Methods
BaggingMargins and out-of-bag observationsPredicting complex skill learning with baggingPredicting heart disease with baggingLimitations of bagging
Boosting
AdaBoost
Predicting atmospheric gamma ray radiation
Predicting complex skill learning with boosting
Limitations of boosting
Random forests
The importance of variables in random forests
Summary
8. Probabilistic Graphical Models
A little graph theory
Bayes' Theorem
Conditional independence
Bayesian networks
The Naïve Bayes classifier
Predicting the sentiment of movie reviews
Hidden Markov models
Predicting promoter gene sequences
Predicting letter patterns in English words
Summary
9. Time Series Analysis
Fundamental concepts of time seriesTime series summary functions
Some fundamental time series
White noiseFitting a white noise time seriesRandom walkFitting a random walk
Stationarity
Stationary time series models
Moving average modelsAutoregressive modelsAutoregressive moving average models
Non-stationary time series models
Autoregressive integrated moving average modelsAutoregressive conditional heteroscedasticity modelsGeneralized autoregressive heteroscedasticity models
Predicting intense earthquakes
Predicting lynx trappings
Predicting foreign exchange rates
Other time series models
Summary
10. Topic Modeling
An overview of topic modeling
Latent Dirichlet Allocation
The Dirichlet distributionThe generative processFitting an LDA model
Modeling the topics of online news stories
Model stabilityFinding the number of topicsTopic distributionsWord distributionsLDA extensions
Summary
11. Recommendation Systems
Rating matrixMeasuring user similarity
Collaborative filtering
User-based collaborative filteringItem-based collaborative filtering
Singular value decomposition
R and Big Data
Predicting recommendations for movies and jokes
Loading and preprocessing the data
Exploring the data
Evaluating binary top-N recommendationsEvaluating non-binary top-N recommendationsEvaluating individual predictions
Other approaches to recommendation systems
Summary
A. Bibliography
Index

Content preview from R: Predictive Analysis

Classification and regression trees and random forest

We will now introduce classification and regression trees (CART) and random forest. CART uses different statistical criteria to decide on tree splits. Random forest uses ensemble learning (a combination of CART trees) to improve classification using a voting principle.

CART

There are a number of differences between CART used for classification and the family of algorithms we just discovered. Here, we only superficially discuss the partitioning criterion and pruning.

In CART, the attribute to be partition is selected with the Gini index as a decision criterion. In classification trees, the Gini index is simply computed as: 1—the sum of the squared probabilities for each possible partition on the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781788290371

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

R: Predictive Analysis

by Tony Fischetti, Eric Mayor, Rui Miguel Forte

Classification and regression trees and random forest

CART

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

More than 5,000 organizations count on O’Reilly

Julian F.

Addison B.

Amir M.

Mark W.

You might also like

R Data Mining

Graphical Data Analysis with R

Mastering Predictive Analytics with R

Quantitative Corpus Linguistics with R

Publisher Resources