book

R: Mining Spatial, Text, Web, and Social Media Data

Name: R: Mining Spatial, Text, Web, and Social Media Data
ISBN: 9781788293747

by Bater Makhabel, Pradeepta Mishra, Nathan Danneman, Richard Heimann

June 2017

Beginner to intermediate

651 pages

14h 24m

English

Packt Publishing

Read now

Unlock full access

R: Mining Spatial, Text, Web, and Social Media Data
Table of Contents
R: Mining Spatial, Text, Web, and Social Media Data
R: Mining Spatial, Text, Web, and Social Media Data
Credits
Preface
What this learning path covers
What you need for this learning path
Who this learning path is for
Reader feedback
Customer support
Downloading the example codeErrataPiracyQuestions

1. Module 1
1. Warming Up
Big dataScalability and efficiency
Data source
Data mining
Feature extractionSummarizationThe data mining processCRISP-DMSEMMA
Social network mining
Social network
Text mining
Information retrieval and text miningMining text for prediction
Web data mining
Why R?
What are the disadvantages of R?
Statistics
Statistics and data miningStatistics and machine learningStatistics and RThe limitations of statistics on data mining
Machine learning
Approaches to machine learningMachine learning architecture
Data attributes and description
Numeric attributesCategorical attributesData descriptionData measuring
Data cleaning
Missing valuesJunk, noisy data, or outlier
Data integration
Data dimension reduction
Eigenvalues and EigenvectorsPrincipal-Component AnalysisSingular-value decompositionCUR decomposition
Data transformation and discretization
Data transformationNormalization data transformation methodsData discretization
Visualization of results
Visualization with R
Time for action
Summary
2. Mining Frequent Patterns, Associations, and Correlations
An overview of associations and patternsPatterns and pattern discoveryThe frequent itemsetThe frequent subsequenceThe frequent substructuresRelationship or rules discoveryAssociation rulesCorrelation rules
Market basket analysis
The market basket modelA-Priori algorithmsInput data characteristics and data structureThe A-Priori algorithmThe R implementationA-Priori algorithm variantsThe Eclat algorithmThe R implementationThe FP-growth algorithmInput data characteristics and data structureThe FP-growth algorithmThe R implementationThe GenMax algorithm with maximal frequent itemsetsThe R implementationThe Charm algorithm with closed frequent itemsetsThe R implementationThe algorithm to generate association rulesThe R implementation
Hybrid association rules mining
Mining multilevel and multidimensional association rulesConstraint-based frequent pattern mining
Mining sequence dataset
Sequence datasetThe GSP algorithm
The R implementation
The SPADE algorithmThe R implementationRule generation from sequential patterns
High-performance algorithms
Time for action
Summary
3. Classification
Classification
Generic decision tree induction
Attribute selection measuresTree pruningGeneral algorithm for the decision tree generationThe R implementation
High-value credit card customers classification using ID3
The ID3 algorithmThe R implementationWeb attack detectionHigh-value credit card customers classification
Web spam detection using C4.5
The C4.5 algorithmThe R implementationA parallel version with MapReduceWeb spam detection
Web key resource page judgment using CART
The CART algorithmThe R implementationWeb key resource page judgment
Trojan traffic identification method and Bayes classification
EstimatingPrior probability estimationLikelihood estimationThe Bayes classificationThe R implementationTrojan traffic identification method
Identify spam e-mail and Naïve Bayes classification
The Naïve Bayes classificationThe R implementationIdentify spam e-mail
Rule-based classification of player types in computer games and rule-based classification
Transformation from decision tree to decision rulesRule-based classificationSequential covering algorithmThe RIPPER algorithmThe R implementationRule-based classification of player types in computer games
Time for action
Summary
4. Advanced Classification
Ensemble (EM) methodsThe bagging algorithmThe boosting and AdaBoost algorithmsThe Random forests algorithmThe R implementationParallel version with MapReduce
Biological traits and the Bayesian belief network
The Bayesian belief network (BBN) algorithmThe R implementationBiological traits
Protein classification and the k-Nearest Neighbors algorithm
The kNN algorithmThe R implementation
Document retrieval and Support Vector Machine
The SVM algorithmThe R implementationParallel version with MapReduceDocument retrieval
Classification using frequent patterns
The associative classificationCBADiscriminative frequent pattern-based classificationThe R implementationText classification using sentential frequent itemsets
Classification using the backpropagation algorithm
The BP algorithmThe R implementationParallel version with MapReduce
Time for action
Summary
5. Cluster Analysis
Search engines and the k-means algorithmThe k-means clustering algorithmThe kernel k-means algorithmThe k-modes algorithmThe R implementationParallel version with MapReduceSearch engine and web page clustering
Automatic abstraction of document texts and the k-medoids algorithm
The PAM algorithmThe R implementationAutomatic abstraction and summarization of document text
The CLARA algorithm
The CLARA algorithmThe R implementation
CLARANS
The CLARANS algorithmThe R implementation
Unsupervised image categorization and affinity propagation clustering
Affinity propagation clusteringThe R implementationUnsupervised image categorizationThe spectral clustering algorithmThe R implementation
News categorization and hierarchical clustering
Agglomerative hierarchical clusteringThe BIRCH algorithmThe chameleon algorithmThe Bayesian hierarchical clustering algorithmThe probabilistic hierarchical clustering algorithmThe R implementationNews categorization
Time for action
Summary
6. Advanced Cluster Analysis
Customer categorization analysis of e-commerce and DBSCANThe DBSCAN algorithmCustomer categorization analysis of e-commerce
Clustering web pages and OPTICS
The OPTICS algorithmThe R implementationClustering web pages
Visitor analysis in the browser cache and DENCLUE
The DENCLUE algorithmThe R implementationVisitor analysis in the browser cache
Recommendation system and STING
The STING algorithmThe R implementationRecommendation systems
Web sentiment analysis and CLIQUE
The CLIQUE algorithmThe R implementationWeb sentiment analysis
Opinion mining and WAVE clustering
The WAVE cluster algorithmThe R implementationOpinion mining
User search intent and the EM algorithm
The EM algorithmThe R implementationThe user search intent
Customer purchase data analysis and clustering high-dimensional data
The MAFIA algorithmThe SURFING algorithmThe R implementationCustomer purchase data analysis
SNS and clustering graph and network data
The SCAN algorithmThe R implementationSocial networking service (SNS)
Time for action
Summary
7. Outlier Detection
Credit card fraud detection and statistical methodsThe likelihood-based outlier detection algorithmThe R implementationCredit card fraud detection
Activity monitoring – the detection of fraud involving mobile phones and proximity-based methods
The NL algorithmThe FindAllOutsM algorithmThe FindAllOutsD algorithmThe distance-based algorithmThe Dolphin algorithmThe R implementationActivity monitoring and the detection of mobile fraud
Intrusion detection and density-based methods
The OPTICS-OF algorithmThe High Contrast Subspace algorithmThe R implementationIntrusion detection
Intrusion detection and clustering-based methods
Hierarchical clustering to detect outliersThe k-means-based algorithmThe ODIN algorithmThe R implementation
Monitoring the performance of the web server and classification-based methods
The OCSVM algorithmThe one-class nearest neighbor algorithmThe R implementationMonitoring the performance of the web server
Detecting novelty in text, topic detection, and mining contextual outliers
The conditional anomaly detection (CAD) algorithmThe R implementationDetecting novelty in text and topic detection
Collective outliers on spatial data
The route outlier detection (ROD) algorithmThe R implementationCharacteristics of collective outliers
Outlier detection in high-dimensional data
The brute-force algorithmThe HilOut algorithmThe R implementation
Time for action
Summary
8. Mining Stream, Time-series, and Sequence Data
The credit card transaction flow and STREAM algorithmThe STREAM algorithmThe single-pass-any-time clustering algorithmThe R implementationThe credit card transaction flow
Predicting future prices and time-series analysis
The ARIMA algorithmPredicting future prices
Stock market data and time-series clustering and classification
The hError algorithmTime-series classification with the 1NN classifierThe R implementationStock market data
Web click streams and mining symbolic sequences
The TECNO-STREAMS algorithmThe R implementationWeb click streams
Mining sequence patterns in transactional databases
The PrefixSpan algorithmThe R implementation
Time for action
Summary
9. Graph Mining and Network Analysis
Graph miningGraphGraph mining algorithms
Mining frequent subgraph patterns
The gPLS algorithmThe GraphSig algorithmThe gSpan algorithmRightmost path extensions and their supportsThe subgraph isomorphism enumeration algorithmThe canonical checking algorithmThe R implementation
Social network mining
Community detection and the shingling algorithmThe node classification and iterative classification algorithmsThe R implementation
Time for action
Summary
10. Mining Text and Web Data
Text mining and TM packages
Text summarization
Topic representationThe multidocument summarization algorithmThe Maximal Marginal Relevance algorithmThe R implementation
The question answering system
Genre categorization of web pages
Categorizing newspaper articles and newswires into topics
The N-gram-based text categorizationThe R implementation
Web usage mining with web logs
The FCA-based association rule mining algorithmThe R implementation
Time for action
Summary
A. Algorithms and Data Structures
2. Module 2
1. Data Manipulation Using In-built R Data
What is data mining?How is it related to data science, analytics, and statistical modeling?
Introduction to the R programming language
Getting started with RData types, vectors, arrays, and matricesList management, factors, and sequencesImport and export of data types
Data type conversion
Sorting and merging dataframes
Indexing or subsetting dataframes
Date and time formatting
Creating new functions
User-defined functionsBuilt-in functions
Loop concepts - the for loop
Loop concepts - the repeat loop
Loop concepts - while conditions
Apply concepts
String manipulation
NA and missing value management
Missing value imputation techniques
Summary
2. Exploratory Data Analysis with Automobile Data
Univariate data analysis
Bivariate analysis
Multivariate analysis
Understanding distributions and transformation
Normal probability distributionBinomial probability distributionPoisson probability distribution
Interpreting distributions
Interpreting continuous data
Variable binning or discretizing continuous data
Contingency tables, bivariate statistics, and checking for data normality
Hypothesis testing
Test of the population meanOne tail test of mean with known varianceOne tail and two tail test of proportionsTwo sample variance test
Non-parametric methods
Wilcoxon signed-rank testMann-Whitney-Wilcoxon testKruskal-Wallis test
Summary
3. Visualize Diamond Dataset
Data visualization using ggplot2Bar chartBoxplotBubble chartDonut chartGeo mappingHistogramLine chartPie chartScatterplotStacked bar chartStem and leaf plotWord cloudCoxcomb plot
Using plotly
Bubble plotBar charts using plotlyScatterplot using plotlyBoxplots using plotlyPolar charts using plotlyPolar scatterplot using plotlyPolar area chart
Creating geo mapping
Summary
4. Regression with Automobile Data
Regression introductionFormulation of regression problemCase study
Linear regression
Stepwise regression method for variable selection
Logistic regression
Cubic regression
Penalized regression
Summary
5. Market Basket Analysis with Groceries Data
Introduction to Market Basket AnalysisWhat is MBA?Where to apply MBA?Data requirementAssumptions/prerequisitesModeling techniquesLimitations
Practical project
Apriori algorithmEclat algorithmVisualizing association rulesImplementation of arules
Summary
6. Clustering with E-commerce Data
Understanding customer segmentationWhy understanding customer segmentation is importantHow to perform customer segmentation?
Various clustering methods available
K-means clusteringHierarchical clusteringModel-based clusteringOther cluster algorithmsComparing clustering methods
References
Summary
7. Building a Retail Recommendation Engine
What is recommendation?Types of product recommendationTechniques to perform recommendation
Assumptions
What method to apply when
Limitations of collaborative filtering
Practical project
Summary
8. Dimensionality Reduction
Why dimensionality reduction?Techniques available for dimensionality reductionWhich technique to apply where?Principal component analysis
Practical project around dimensionality reduction
Attribute description
Parametric approach to dimension reduction
References
Summary
9. Applying Neural Network to Healthcare Data
Introduction to neural networks
Understanding the math behind the neural network
Neural network implementation in R
Neural networks for prediction
Neural networks for classification
Neural networks for forecasting
Merits and demerits of neural networks
References
Summary
3. Module 3
1. Going Viral
Social media mining using sentiment analysis
The state of communication
What is Big Data?
Human sensors and honest signals
Quantitative approaches
Summary
2. Getting Started with R
Why R?
Quick start
The basics – assignment and arithmeticFunctions, arguments, and help
Vectors, sequences, and combining vectors
A quick example – creating data frames and importing files
Visualization in R
Style and workflow
Additional resources
Summary
3. Mining Twitter with R
Why Twitter data?
Obtaining Twitter data
Preliminary analyses
Summary
4. Potentials and Pitfalls of Social Media Data
Opinion mining made difficult
Sentiment and its measurement
The nature of social media data
Traditional versus nontraditional social data
Measurement and inferential challenges
Summary
5. Social Media Mining – Fundamentals
Key concepts of social media mining
Good data versus bad data
Understanding sentiments
Scherer's typology of emotions
Sentiment polarity – data and classification
Supervised social media mining – lexicon-based sentiment
Supervised social media mining – Naive Bayes classifiers
Unsupervised social media mining – Item Response Theory for text scaling
Summary
6. Social Media Mining – Case Studies
Introductory considerations
Case study 1 – supervised social media mining – lexicon-based sentiment
Case study 2 – Naive Bayes classifier
Case study 3 – IRT models for unsupervised sentiment scaling
Summary
A. Conclusions and Next Steps
Final thoughts
An expanding field
Further reading
Bibliography
Bibliography
Index

Content preview from R: Mining Spatial, Text, Web, and Social Media Data

Trojan traffic identification method and Bayes classification

Among probabilistic classification algorithms is the Bayes classification, which is based on Bayes' theorem. It predicts the instance or the class as the one that makes the posterior probability maximal. The risk for Bayes classification is that it needs enough data to estimate the joint probability density more reliably.

Given a dataset D with a size n, each instance or point x belonging to D with a dimension of m, for each . To predict the class of any x, we use the following formula: ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781788293747

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

R: Mining Spatial, Text, Web, and Social Media Data

by Bater Makhabel, Pradeepta Mishra, Nathan Danneman, Richard Heimann

Trojan traffic identification method and Bayes classification

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

More than 5,000 organizations count on O’Reilly

Julian F.

Addison B.

Amir M.

Mark W.

You might also like

R Data Mining

Transfer Learning for Natural Language Processing

Sentiment Analysis in Social Networks

Develop Intelligent iOS Apps with Swift: Understand Texts, Classify Sentiments, and Autodetect Answers in Text Using NLP

Publisher Resources