book

R: Recipes for Analysis, Visualization and Machine Learning

Name: R: Recipes for Analysis, Visualization and Machine Learning
ISBN: 9781787289598

by Viswa Viswanathan, Shanthi Viswanathan, Atmajitsinh Gohil, Chiu Yu-Wei

November 2016

Beginner to intermediate

959 pages

43h 46m

English

Packt Publishing

Read now

Unlock full access

R: Recipes for Analysis, Visualization and Machine Learning
Table of Contents
R: Recipes for Analysis, Visualization and Machine Learning
R: Recipes for Analysis, Visualization and Machine Learning
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions

1. Module 1
1. A Simple Guide to R
Installing packages and getting help in RGetting readyHow to do it…How it works…There's more…See also
Data types in R
How to do it…
Special values in R
How to do it…How it works…
Matrices in R
How to do it…How it works…
Editing a matrix in R
How to do it…
Data frames in R
How to do it…
Editing a data frame in R
How to do it...
Importing data in R
How to do it...How it works…
Exporting data in R
How to do it…How it works…
Writing a function in R
Getting readyHow to do it…How it works…See also
Writing if else statements in R
How to do it…How it works…
Basic loops in R
How to do it…How it works…
Nested loops in R
How to do it…
The apply, lapply, sapply, and tapply functions
How to do it…How it works…
Using par to beautify a plot in R
How to do it…How it works…
Saving plots
How to do it…How it works…
2. Practical Machine Learning with R
Introduction
Downloading and installing R
Getting readyHow to do it...How it works...See also
Downloading and installing RStudio
Getting readyHow to do it...How it worksSee also
Installing and loading packages
Getting readyHow to do it...How it worksSee also
Reading and writing data
Getting readyHow to do it...How it worksSee also
Using R to manipulate data
Getting readyHow to do it...How it worksThere's more...
Applying basic statistics
Getting readyHow to do it...How it works...There's more...
Visualizing data
Getting readyHow to do it...How it works...See also
Getting a dataset for machine learning
Getting readyHow to do it...How it works...See also
3. Acquire and Prepare the Ingredients – Your Data
Introduction
Reading data from CSV files
Getting readyHow to do it...How it works...There's more...Handling different column delimitersHandling column headers/variable namesHandling missing valuesReading strings as characters and not as factorsReading data directly from a website
Reading XML data
Getting readyHow to do it...How it works...There's more...Extracting HTML table data from a web pageExtracting a single HTML table from a web page
Reading JSON data
Getting readyHow to do it...How it works...
Reading data from fixed-width formatted files
Getting readyHow to do it...How it works...There's more...Files with headersExcluding columns from data
Reading data from R files and R libraries
Getting readyHow to do it...How it works...There's more...To save all objects in a sessionTo selectively save objects in a sessionAttaching/detaching R data files to an environmentListing all datasets in loaded packages
Removing cases with missing values
Getting readyHow to do it...How it works...There's more...Eliminating cases with NA for selected variablesFinding cases that have no missing valuesConverting specific values to NAExcluding NA values from computations
Replacing missing values with the mean
Getting readyHow to do it...How it works...There's more...Imputing random values sampled from nonmissing values
Removing duplicate cases
Getting readyHow to do it...How it works...There's more...Identifying duplicates (without deleting them)
Rescaling a variable to [0,1]
Getting readyHow to do it...How it works...There's more...Rescaling many variables at onceSee also…
Normalizing or standardizing data in a data frame
Getting readyHow to do it...How it works...There's more...Standardizing several variables simultaneouslySee also…
Binning numerical data
Getting readyHow to do it...How it works...There's more...Creating a specified number of intervals automatically
Creating dummies for categorical variables
Getting readyHow to do it...How it works...There's more...Choosing which variables to create dummies for
4. What's in There? – Exploratory Data Analysis
Introduction
Creating standard data summaries
Getting readyHow to do it...How it works...There's more...Using the str() function for an overview of a data frameComputing the summary for a single variableFinding the mean and standard deviation
Extracting a subset of a dataset
Getting readyHow to do it...How it works...There's more...Excluding columnsSelecting based on multiple valuesSelecting using logical vector
Splitting a dataset
Getting readyHow to do it...How it works...
Creating random data partitions
Getting readyHow to do it…Case 1 – numerical target variable and two partitionsCase 2 – numerical target variable and three partitionsCase 3 – categorical target variable and two partitionsCase 4 – categorical target variable and three partitionsHow it works...There's more...Using a convenience function for partitioningSampling from a set of values
Generating standard plots such as histograms, boxplots, and scatterplots
Getting readyHow to do it...HistogramsBoxplotsScatterplotsScatterplot matricesHow it works...HistogramsBoxplotsThere's more...Overlay a density plot on a histogramOverlay a regression line on a scatterplotColor specific points on a scatterplot
Generating multiple plots on a grid
Getting readyHow to do it...How it works...Graphics parametersSee also…
Selecting a graphics device
Getting readyHow to do it...How it works...See also…
Creating plots with the lattice package
Getting readyHow to do it...How it works...There's more...Adding flair to your graphsSee also…
Creating plots with the ggplot2 package
Getting readyHow to do it...How it works...There's more...Graph using qplotCondition plots on continuous numeric variablesSee also…
Creating charts that facilitate comparisons
Getting readyHow to do it...Using base plotting systemUsing ggplot2How it works...There's more...Creating boxplots with ggplot2See also…
Creating charts that help visualize a possible causality
Getting readyHow to do it...See also…
Creating multivariate plots
Getting readyHow to do it...How it works...See also…
5. Where Does It Belong? – Classification
Introduction
Generating error/classification-confusion matrices
Getting readyHow to do it...How it works...There's more...Visualizing the error/classification confusion matrixComparing the model's performance for different classes
Generating ROC charts
Getting readyHow to do it...How it works...There's more…Using arbitrary class labels
Building, plotting, and evaluating – classification trees
Getting readyHow to do it...How it works...There's more...Computing raw probabilitiesCreate the ROC ChartSee also
Using random forest models for classification
Getting readyHow to do it...How it works...There's more...Computing raw probabilitiesGenerating the ROC chartSpecifying cutoffs for classificationSee also...
Classifying using Support Vector Machine
Getting readyHow to do it...How it works...There's more...Controlling scaling of variablesDetermining the type of SVM modelAssigning weights to the classesSee also...
Classifying using the Naïve Bayes approach
Getting readyHow to do it...How it works...See also...
Classifying using the KNN approach
Getting readyHow to do it...How it works...There's more...Automating the process of running KNN for many k valuesUsing KNN to compute raw probabilities instead of classifications
Using neural networks for classification
Getting readyHow to do it...How it works...There's more...Exercising greater control over nnetGenerating raw probabilities
Classifying using linear discriminant function analysis
Getting readyHow to do it...How it works...There's more...Using the formula interface for ldaSee also ...
Classifying using logistic regression
Getting readyHow to do it...How it works...
Using AdaBoost to combine classification tree models
Getting readyHow to do it...How it works...
6. Give Me a Number – Regression
Introduction
Computing the root mean squared error
Getting readyHow to do it...How it works...There's more...Using a convenience function to compute the RMS error
Building KNN models for regression
Getting readyHow to do it...How it works...There's more...Running KNN with cross-validation in place of validation partitionUsing a convenience function to run KNNUsing a convenience function to run KNN for multiple k valuesSee also...
Performing linear regression
Getting readyHow to do it...How it works...There's more...Forcing lm to use a specific factor level as the referenceUsing other options in the formula expression for linear modelsSee also...
Performing variable selection in linear regression
Getting readyHow to do it...How it works...See also...
Building regression trees
Getting readyHow to do it...How it works...There's more…Generating regression trees for data with categorical predictorsSee also...
Building random forest models for regression
Getting readyHow to do it...How it works...There's more...Controlling forest generationSee also...
Using neural networks for regression
Getting readyHow to do it...How it works...See also...
Performing k-fold cross-validation
Getting readyHow to do it...How it works...See also...
Performing leave-one-out-cross-validation to limit overfitting
How to do it...How it works...See also...
7. Can You Simplify That? – Data Reduction Techniques
Introduction
Performing cluster analysis using K-means clustering
Getting readyHow to do it...How it works...There's more...Use a convenience function to choose a value for KSee also...
Performing cluster analysis using hierarchical clustering
Getting readyHow to do it...How it works...See also...
Reducing dimensionality with principal component analysis
Getting readyHow to do it...How it works...
8. Lessons from History – Time Series Analysis
Introduction
Creating and examining date objects
Getting readyHow to do it...How it works...See also...
Operating on date objects
Getting readyHow to do it...How it works...See also...
Performing preliminary analyses on time series data
Getting readyHow to do it...How it works...See also...
Using time series objects
Getting readyHow to do it...How it works...See also...
Decomposing time series
Getting readyHow to do it...How it works...See also...
Filtering time series data
Getting readyHow to do it...How it works...See also...
Smoothing and forecasting using the Holt-Winters method
Getting readyHow to do it...How it works...See also...
Building an automated ARIMA model
Getting readyHow to do it...How it works...See also...
9. It's All About Your Connections – Social Network Analysis
Introduction
Downloading social network data using public APIs
Getting readyHow to do it...How it works...See also...
Creating adjacency matrices and edge lists
Getting readyHow to do it...How it works...See also...
Plotting social network data
Getting readyHow to do it...How it works...There's more...Specifying plotting preferencesPlotting directed graphsCreating a graph object with weightsExtracting the network as an adjacency matrix from the graph objectExtracting an adjacency matrix with weightsExtracting edge list from graph objectCreating bipartite network graphGenerating projections of a bipartite networkSee also...
Computing important network metrics
Getting readyHow to do it...How it works...There's more...Getting edge sequencesGetting immediate and distant neighborsAdding vertices or nodesAdding edgesDeleting isolates from a graphCreating subgraphs
10. Put Your Best Foot Forward – Document and Present Your Analysis
Introduction
Generating reports of your data analysis with R Markdown and knitr
Getting readyHow to do it...How it works...There's more...Using the render functionAdding output options
Creating interactive web applications with shiny
Getting readyHow to do it...How it works...There's more...Adding imagesAdding HTMLAdding tab setsAdding a dynamic UICreating single file web application
Creating PDF presentations of your analysis with R Presentation
Getting readyHow to do it...How it works...There's more...Using hyperlinksControlling the displayEnhancing the look of the presentation
11. Work Smarter, Not Harder – Efficient and Elegant R Code
Introduction
Exploiting vectorized operations
Getting readyHow to do it...How it works...There's more...
Processing entire rows or columns using the apply function
Getting readyHow to do it...How it works...There's more...Using apply on a three-dimensional array
Applying a function to all elements of a collection with lapply and sapply
Getting readyHow to do it...How it works...There's more...Dynamic outputOne caution
Applying functions to subsets of a vector
Getting readyHow to do it...How it works...There's more...Applying a function on groups from a data frame
Using the split-apply-combine strategy with plyr
Getting readyHow to do it...How it works...There's more...Adding a new column using transformUsing summarize along with the plyr functionConcatenating the list of data frames into a big data frame
Slicing, dicing, and combining data with data tables
Getting readyHow to do it...How it works...There's more...Adding multiple aggregated columnsCounting groupsDeleting a columnJoining data tablesUsing symbols
12. Where in the World? – Geospatial Analysis
Introduction
Downloading and plotting a Google map of an area
Getting readyHow to do it...How it works...There's more...Saving the downloaded map as an image fileGetting a satellite image
Overlaying data on the downloaded Google map
Getting readyHow to do it...How it works...
Importing ESRI shape files into R
Getting readyHow to do it...How it works...
Using the sp package to plot geographic data
Getting readyHow to do it...How it works...
Getting maps from the maps package
Getting readyHow to do it...How it works...
Creating spatial data frames from regular data frames containing spatial and other data
Getting readyHow to do it...How it works...
Creating spatial data frames by combining regular data frames with spatial objects
Getting readyHow to do it...How it works...
Adding variables to an existing spatial data frame
Getting readyHow to do it...How it works...
13. Playing Nice – Connecting to Other Systems
Introduction
Using Java objects in R
Getting readyHow to do it...How it works...There's more...Checking JVM propertiesDisplaying available methods
Using JRI to call R functions from Java
Getting readyHow to do it...How it works...There's more...
Using Rserve to call R functions from Java
Getting readyHow to do it...How it works...There's more...Retrieving an array from R
Executing R scripts from Java
Getting readyHow to do it...How it works...
Using the xlsx package to connect to Excel
Getting readyHow to do it...How it works...
Reading data from relational databases – MySQL
Getting readyHow to do it...Using RODBCUsing RMySQLUsing RJDBCHow it works...Using RODBCUsing RMySQLUsing RJDBCThere's more...Fetching all rowsWhen the SQL query is long
Reading data from NoSQL databases – MongoDB
Getting readyHow to do it...How it works...There's more...Validating your JSON
2. Module 2
1. Basic and Interactive Plots
Introduction
Introducing a scatter plot
Getting readyHow to do it…How it works…
Scatter plots with texts, labels, and lines
How to do it…How it works…There's more…See also
Connecting points in a scatter plot
How to do it…How it works…There's more…See also
Generating an interactive scatter plot
Getting readyHow to do it…How it works…There's more…See also
A simple bar plot
How to do it…How it works…There's more…See also
An interactive bar plot
Getting readyHow to do it…How it works…There's more…See also
A simple line plot
Getting readyHow to do it…How it works…See also
Line plot to tell an effective story
Getting readyHow to do it…How it works…See also
Generating an interactive Gantt/timeline chart in R
Getting readyHow to do it…See also
Merging histograms
How to do it…How it works…
Making an interactive bubble plot
How to do it…How it works…There's more…See also
Constructing a waterfall plot in R
Getting readyHow to do it…
2. Heat Maps and Dendrograms
Introduction
Constructing a simple dendrogram
Getting readyHow to do it…How it works…There's more...See also
Creating dendrograms with colors and labels
Getting readyHow to do it…How it works…There's more…
Creating a heat map
Getting readyHow to do it…How it works…There's more…See also
Generating a heat map with customized colors
Getting readyHow to do it…How it works…
Generating an integrated dendrogram and a heat map
How to do it…There's more…See also
Creating a three-dimensional heat map and a stereo map
Getting readyHow to do it…See also
Constructing a tree map in R
Getting readyHow to do it…How it works…There's more…See also
3. Maps
Introduction
Introducing regional maps
Getting readyHow to do it…How it works…See also
Introducing choropleth maps
Getting readyHow to do it…How it works…There's more…See also
A guide to contour maps
How to do it…How it works…There's more…See also
Constructing maps with bubbles
Getting readyHow to do it…How it works...There's more…See also
Integrating text with maps
Getting readyHow to do it…See also
Introducing shapefiles
Getting readyHow to do it…See also
Creating cartograms
Getting readyHow to do it…See also
4. The Pie Chart and Its Alternatives
Introduction
Generating a simple pie chart
How to do it…How it works…There's more...See also
Constructing pie charts with labels
Getting readyHow to do it…How it works…There's more…
Creating donut plots and interactive plots
Getting radyHow to do it...How it works…There's more…See also
Generating a slope chart
Getting readyHow to do it…How it works…See also
Constructing a fan plot
Getting readyHow to do it…How it works…
5. Adding the Third Dimension
Introduction
Constructing a 3D scatter plot
Getting readyHow to do it…How it works…There's more…See also
Generating a 3D scatter plot with text
Getting readyHow to do it…How it works…There's more…See also
A simple 3D pie chart
Getting readyHow to do it…How it works…
A simple 3D histogram
Getting readyHow to do it…How it works…There's more...
Generating a 3D contour plot
Getting readyHow to do it…How it works…
Integrating a 3D contour and a surface plot
Getting readyHow to do it…How it works…There's more...See also
Animating a 3D surface plot
Getting readyHow to do it…How it works…There's more…See also
6. Data in Higher Dimensions
Introduction
Constructing a sunflower plot
Getting readyHow to do it…How it works…See also
Creating a hexbin plot
Getting readyHow to do it…How it works…See also
Generating interactive calendar maps
Getting readyHow to do it…How it works…See also
Creating Chernoff faces in R
Getting readyHow to do it…How it works…
Constructing a coxcomb plot in R
Getting readyHow to do it…How it works…See also
Constructing network plots
Getting readyHow to do it…How it works…There's more…See also
Constructing a radial plot
Getting readyHow to do it…How it works…There's more…See also
Generating a very basic pyramid plot
Getting readyHow to do it…How it works…See also
7. Visualizing Continuous Data
Introduction
Generating a candlestick plot
Getting readyHow to do it…How it works…There's more…See also
Generating interactive candlestick plots
Getting readyHow to do it…How it works…
Generating a decomposed time series
How to do it…How it works…There's more…See also
Plotting a regression line
How to do it…How it works…See also
Constructing a box and whiskers plot
Getting readyHow to do it…How it works…See also
Generating a violin plot
Getting readyHow to do it…
Generating a quantile-quantile plot (QQ plot)
Getting readyHow to do it…See also
Generating a density plot
Getting readyHow to do it…How it works…There's more…See also
Generating a simple correlation plot
Getting readyHow to do it…How it works…There's more…See also
8. Visualizing Text and XKCD-style Plots
Introduction
Generating a word cloud
Getting readyHow to do it…How it works…There's more…See also
Constructing a word cloud from a document
Getting readyHow to do it…How it works…There's more…See also
Generating a comparison cloud
Getting readyHow to do it…How it works…See also
Constructing a correlation plot and a phrase tree
Getting readyHow to do it…How it works…There's more…See also
Generating plots with custom fonts
Getting readyHow to do it…How it works…See also
Generating an XKCD-style plot
Getting readyHow to do it…See also
9. Creating Applications in R
Introduction
Creating animated plots in R
Getting readyHow to do it…How it works…
Creating a presentation in R
Getting readyHow to do it…How it works…There's more…See also
A basic introduction to API and XML
Getting readyHow to do it…How it works…See also
Constructing a bar plot using XML in R
Getting readyHow to do it…How it works…See also
Creating a very simple shiny app in R
Getting readyHow to do it…How it works…See also
3. Module 3
1. Data Exploration with RMS Titanic
Introduction
Reading a Titanic dataset from a CSV file
Getting readyHow to do it...How it works...There's more...
Converting types on character variables
Getting readyHow to do it...How it works...There's more...
Detecting missing values
Getting readyHow to do it...How it works...There's more...
Imputing missing values
Getting readyHow to do it...How it works...There's more...
Exploring and visualizing data
Getting readyHow to do it...How it works...There's more...See also
Predicting passenger survival with a decision tree
Getting readyHow to do it...How it works...There's more...
Validating the power of prediction with a confusion matrix
Getting readyHow to do it...How it works...There's more...
Assessing performance with the ROC curve
Getting readyHow to do it...How it works...See also
2. R and Statistics
Introduction
Understanding data sampling in R
Getting readyHow to do it...How it works...See also
Operating a probability distribution in R
Getting readyHow to do it...How it works...There's more...
Working with univariate descriptive statistics in R
Getting readyHow to do it...How it works...There's more...
Performing correlations and multivariate analysis
Getting readyHow to do it...How it works...See also
Operating linear regression and multivariate analysis
Getting readyHow to do it...How it works...See also
Conducting an exact binomial test
Getting readyHow to do it...How it works...See also
Performing student's t-test
Getting readyHow to do it...How it works...See also
Performing the Kolmogorov-Smirnov test
Getting readyHow to do it...How it works...See also
Understanding the Wilcoxon Rank Sum and Signed Rank test
Getting readyHow to do it...How it works...See also
Working with Pearson's Chi-squared test
Getting readyHow to do itHow it works...There's more...
Conducting a one-way ANOVA
Getting readyHow to do it...How it works...There's more...
Performing a two-way ANOVA
Getting readyHow to do it...How it works...See also
3. Understanding Regression Analysis
Introduction
Fitting a linear regression model with lm
Getting readyHow to do it...How it works...There's more...
Summarizing linear model fits
Getting readyHow to do it...How it works...See also
Using linear regression to predict unknown values
Getting readyHow to do it...How it works...See also
Generating a diagnostic plot of a fitted model
Getting readyHow to do it...How it works...There's more...
Fitting a polynomial regression model with lm
Getting readyHow to do it...How it worksThere's more...
Fitting a robust linear regression model with rlm
Getting readyHow to do it...How it worksThere's more...
Studying a case of linear regression on SLID data
Getting readyHow to do it...How it works...See also
Applying the Gaussian model for generalized linear regression
Getting readyHow to do it...How it works...See also
Applying the Poisson model for generalized linear regression
Getting readyHow to do it...How it works...See also
Applying the Binomial model for generalized linear regression
Getting readyHow to do it...How it works...See also
Fitting a generalized additive model to data
Getting readyHow to do it...How it worksSee also
Visualizing a generalized additive model
Getting readyHow to do it...How it works...There's more...
Diagnosing a generalized additive model
Getting readyHow to do it...How it works...There's more...
4. Classification (I) – Tree, Lazy, and Probabilistic
Introduction
Preparing the training and testing datasets
Getting readyHow to do it...How it works...There's more...
Building a classification model with recursive partitioning trees
Getting readyHow to do it...How it works...See also
Visualizing a recursive partitioning tree
Getting readyHow to do it...How it works...See also
Measuring the prediction performance of a recursive partitioning tree
Getting readyHow to do it...How it works...See also
Pruning a recursive partitioning tree
Getting readyHow to do it...How it works...See also
Building a classification model with a conditional inference tree
Getting readyHow to do it...How it works...See also
Visualizing a conditional inference tree
Getting readyHow to do it...How it works...See also
Measuring the prediction performance of a conditional inference tree
Getting readyHow to do it...How it works...See also
Classifying data with the k-nearest neighbor classifier
Getting readyHow to do it...How it works...See also
Classifying data with logistic regression
Getting readyHow to do it...How it works...See also
Classifying data with the Naïve Bayes classifier
Getting readyHow to do it...How it works...See also
5. Classification (II) – Neural Network and SVM
Introduction
Classifying data with a support vector machine
Getting readyHow to do it...How it works...See also
Choosing the cost of a support vector machine
Getting readyHow to do it...How it works...See also
Visualizing an SVM fit
Getting readyHow to do it...How it works...See also
Predicting labels based on a model trained by a support vector machine
Getting readyHow to do it...How it works...There's more...
Tuning a support vector machine
Getting readyHow to do it...How it works...See also
Training a neural network with neuralnet
Getting readyHow to do it...How it works...See also
Visualizing a neural network trained by neuralnet
Getting readyHow to do it...How it works...See also
Predicting labels based on a model trained by neuralnet
Getting readyHow to do it...How it works...See also
Training a neural network with nnet
Getting readyHow to do it...How it works...See also
Predicting labels based on a model trained by nnet
Getting readyHow to do it...How it works...See also
6. Model Evaluation
Introduction
Estimating model performance with k-fold cross-validation
Getting readyHow to do it...How it works...There's more...
Performing cross-validation with the e1071 package
Getting readyHow to do it...How it works...See also
Performing cross-validation with the caret package
Getting readyHow to do it...How it works...See also
Ranking the variable importance with the caret package
Getting readyHow to do it...How it works...There's more...
Ranking the variable importance with the rminer package
Getting readyHow to do it...How it works...See also
Finding highly correlated features with the caret package
Getting readyHow to do it...How it works...See also
Selecting features using the caret package
Getting readyHow to do it...How it works...See also
Measuring the performance of the regression model
Getting readyHow to do it...How it works...There's more…
Measuring prediction performance with a confusion matrix
Getting readyHow to do it...How it works...See also
Measuring prediction performance using ROCR
Getting readyHow to do it...How it works...See also
Comparing an ROC curve using the caret package
Getting readyHow to do it...How it works...See also
Measuring performance differences between models with the caret package
Getting readyHow to do it...How it works...See also
7. Ensemble Learning
Introduction
Classifying data with the bagging method
Getting readyHow to do it...How it works...There's more...
Performing cross-validation with the bagging method
Getting readyHow to do it...How it works...See also
Classifying data with the boosting method
Getting readyHow to do it...How it works...There's more...
Performing cross-validation with the boosting method
Getting readyHow to do it...How it works...See also
Classifying data with gradient boosting
Getting readyHow to do it...How it works...There's more...
Calculating the margins of a classifier
Getting readyHow to do it...How it works...See also
Calculating the error evolution of the ensemble method
Getting readyHow to do it...How it works...See also
Classifying data with random forest
Getting readyHow to do it...How it works...There's more...
Estimating the prediction errors of different classifiers
Getting readyHow to do it...How it works...See also
8. Clustering
Introduction
Clustering data with hierarchical clustering
Getting readyHow to do it...How it works...There's more...
Cutting trees into clusters
Getting readyHow to do it...How it works...There's more...
Clustering data with the k-means method
Getting readyHow to do it...How it works...See also
Drawing a bivariate cluster plot
Getting readyHow to do it...How it works...There's more
Comparing clustering methods
Getting readyHow to do it...How it works...See also
Extracting silhouette information from clustering
Getting readyHow to do it...How it works...See also
Obtaining the optimum number of clusters for k-means
Getting readyHow to do it...How it works...See also
Clustering data with the density-based method
Getting readyHow to do it...How it works...See also
Clustering data with the model-based method
Getting readyHow to do it...How it works...See also
Visualizing a dissimilarity matrix
Getting readyHow to do it...How it works...There's more...
Validating clusters externally
Getting readyHow to do it...How it works...See also
9. Association Analysis and Sequence Mining
Introduction
Transforming data into transactions
Getting readyHow to do it...How it works...See also
Displaying transactions and associations
Getting readyHow to do it...How it works...See also
Mining associations with the Apriori rule
Getting readyHow to do it...How it works...See also
Pruning redundant rules
Getting readyHow to do it...How it works...See also
Visualizing association rules
Getting readyHow to do it...How it works...See also
Mining frequent itemsets with Eclat
Getting readyHow to do it...How it works...See also
Creating transactions with temporal information
Getting readyHow to do it...How it works...See also
Mining frequent sequential patterns with cSPADE
Getting readyHow to do it...How it works...See also
10. Dimension Reduction
Introduction
Performing feature selection with FSelector
Getting readyHow to do it...How it works...See also
Performing dimension reduction with PCA
Getting readyHow to do it...How it works...There's more...
Determining the number of principal components using the scree test
Getting readyHow to do it...How it works...There's more...
Determining the number of principal components using the Kaiser method
Getting readyHow to do it...How it works...See also
Visualizing multivariate data using biplot
Getting readyHow to do it...How it works...There's more...
Performing dimension reduction with MDS
Getting readyHow to do it...How it works...There's more...
Reducing dimensions with SVD
Getting readyHow to do it...How it works...See also
Compressing images with SVD
Getting readyHow to do it...How it works...See also
Performing nonlinear dimension reduction with ISOMAP
Getting readyHow to do it...How it works...There's more...
Performing nonlinear dimension reduction with Local Linear Embedding
Getting readyHow to do it...How it works...See also
11. Big Data Analysis (R and Hadoop)
Introduction
Preparing the RHadoop environment
Getting readyHow to do it...How it works...See also
Installing rmr2
Getting readyHow to do it...How it works...See also
Installing rhdfs
Getting readyHow to do it...How it works...See also
Operating HDFS with rhdfs
Getting readyHow to do it...How it works...See also
Implementing a word count problem with RHadoop
Getting readyHow to do it...How it works...See also
Comparing the performance between an R MapReduce program and a standard R program
Getting readyHow to do it...How it works...See also
Testing and debugging the rmr2 program
Getting readyHow to do it...How it works...See also
Installing plyrmr
Getting readyHow to do it...How it works...See also
Manipulating data with plyrmr
Getting readyHow to do it...How it works...See also
Conducting machine learning with RHadoop
Getting readyHow to do it...How it works...See also
Configuring RHadoop clusters on Amazon EMR
Getting readyHow to do it...How it works...See also
A. Resources for R and Machine Learning
B. Dataset – Survival of Passengers on the Titanic
A. Bibliography
Index

Content preview from R: Recipes for Analysis, Visualization and Machine Learning

Operating HDFS with rhdfs

The rhdfs package is an interface between Hadoop and R, which can call an HDFS API in the backend to operate HDFS. As a result, you can easily operate HDFS from the R console through the use of the rhdfs package. In the following recipe, we will demonstrate how to use the rhdfs function to manipulate HDFS.

Getting ready

To proceed with this recipe, you need to have completed the previous recipe by installing rhdfs into R, and validate that you can initial HDFS via the hdfs.init function.

How to do it...

Perform the following steps to operate files stored on HDFS:

Initialize the rhdfs package:

> Sys.setenv(HADOOP_CMD="/usr/bin/hadoop")
> Sys.setenv(HADOOP_STREAMING="/usr/lib/hadoop-mapreduce/hadoop-streaming-2.5.0-cdh5.2.0.jar") ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

R: Data Analysis and Visualization

Tony Fischetti, Brett Lantz, Jaynal Abedin, Hrishi V. Mittal, Bater Makhabel, Edina Berlinger, Ferenc Illés, Milán Badics, Ádám Banai, Gergely Daróczi, Barbara Dömötör, Gergely Gabler, Dániel Havran, Péter Juhász, István Margitai, Balázs Márkus, Péter Medvegyev, Julia Molnár, Balázs Árpád Szucs, Ágnes Tuza, Tamás Vadász, Kata Váradi, Ágnes Vidovics-Dancs

Advanced R Statistical Programming and Data Models: Analysis, Machine Learning, and Visualization

Publisher Resources

ISBN: 9781787289598Purchase Link

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills