book

Data Science in R

by Deborah Nolan

April 2015

Beginner to intermediate

539 pages

15h 21m

English

Chapman and Hall/CRC

Read now

Unlock full access

Goals of the BookUsing These Case Studies in Statistical Computing CoursesBroad TopicsTarget AudienceThe Themes of the Three PartsTypographic ConventionsAvailable Materials
1.1 Introduction1.1.1 Computational Topics1.2 The Raw Data1.2.1 Processing the Raw Data1.3 Cleaning the Data and Building a Representation for Analysis1.3.1 Exploring Orientation1.3.2 Exploring MAC Addresses1.3.3 Exploring the Position of the Hand-Held Device1.3.4 Creating a Function to Prepare the Data1.4 Signal Strength Analysis1.4.1 Distribution of Signal Strength1.4.2 The Relationship between Signal and Distance1.5 Nearest Neighbor Methods to Predict Location1.5.1 Preparing the Test Data1.5.2 Choice of Orientation1.5.3 Finding the Nearest Neighbors1.5.4 Cross-Validation and Choice of k1.6 ExercisesBibliographyFigure 1.1Figure 1.2Figure 1.3Figure 1.4Figure 1.5Figure 1.6Figure 1.7Figure 1.8Figure 1.9Figure 1.10Figure 1.11Figure 1.12Figure 1.13Table 1.1
2.1 Introduction2.1.1 Computational Topics2.2 Reading Tables of Race Results into R2.3 Data Cleaning and Reformatting Variables2.4 Exploring the Run Time for All Male Runners2.4.1 Making Plots with Many Observations2.4.2 Fitting Models to Average Performance2.4.3 Cross-Sectional Data and Covariates2.5 Constructing a Record for an Individual Runner across Years2.6 Modeling the Change in Running Time for Individuals2.7 Scraping Race Results from the Web2.8 ExercisesBibliographyFigure 2.1Figure 2.2Figure 2.3Figure 2.4Figure 2.5Figure 2.6Figure 2.7Figure 2.8Figure 2.9Figure 2.10Figure 2.11Figure 2.12Figure 2.13Figure 2.14Figure 2.15Figure 2.16Figure 2.17Figure 2.18Figure 2.19Figure 2.20Figure 2.21

3.1 Introduction3.1.1 Computational Topics3.2 Anatomy of an email Message3.3 Reading the email Messages3.4 Text Mining and Naïve Bayes Classification3.5 Finding the Words in a Message3.5.1 Splitting the Message into Its Header and Body3.5.2 Removing Attachments from the Message Body3.5.3 Extracting Words from the Message Body3.5.4 Completing the Data Preparation Process3.6 Implementing the Naïve Bayes Classifier3.6.1 Test and Training Data3.6.2 Probability Estimates from Training Data3.6.3 Classifying New Messages3.6.4 Computational Considerations3.7 Recursive Partitioning and Classification Trees3.8 Organizing an email Message into an R Data Structure3.8.1 Processing the Header3.8.2 Processing Attachments3.8.3 Testing Our Code on More email Data3.8.4 Completing the Process3.9 Deriving Variables from the email Message3.9.1 Checking Our Code for Errors3.10 Exploring the email Feature Set3.11 Fitting the rpart() Model to the email Data3.12 ExercisesBibliographyFigure 3.1Figure 3.2Figure 3.3Figure 3.4Figure 3.5Figure 3.6Figure 3.7Figure 3.8Figure 3.9Table 3.1
4.1 Description4.1.1 Computational Topics4.2 The Data4.2.1 Reading an Entire Log File4.2.2 Exploring Log Files4.2.3 Visualizing the Path4.2.4 Exploring a "Look"4.2.5 The Error Distribution for Range Values4.3 Detecting a Circular Target4.3.1 Connecting Segments Behind the Robot4.3.2 Determining If a Segment Corresponds to a Circle4.4 Detecting the Target with Streaming Data in Real TimeBibliographyFigure 4.1Figure 4.2Figure 4.3Figure 4.4Figure 4.5Figure 4.6Figure 4.7Figure 4.8Figure 4.9Figure 4.10Figure 4.11Figure 4.12Figure 4.13Figure 4.14Figure 4.15Figure 4.16
5.1 Introduction5.1.1 Computational Topics5.2 Acquiring the Airline Data Set5.3 Computing with Massive Data: Getting Flight Delay Counts5.3.1 The R Programming Environment5.3.2 The UNIX Shell5.3.3 An SQL Database with R5.3.4 The bigmemory Package with R5.4 Explorations Using Parallel Computing: The Distribution of Flight Delays5.4.1 Writing a Parallelizable Loop with foreach5.4.2 Using the Split-Apply-Combine Approach for Better Performance5.4.3 Using Split-Apply-Combine to Find the Best Time to Fly5.5 From Exploration to Model: Do Older Planes Suffer Greater Delays?BibliographyFigure 5.1
6.1 The Problem6.1.1 Computational Topics6.2 The Data Format6.3 Reading the Financial Data6.4 Visualizing the Time Series6.5 Finding Opening and Closing Positions6.5.1 Identifying a Position6.5.2 Displaying Positions6.5.3 Finding All Positions6.5.4 Computing the Profit for a Position6.5.5 Finding the Optimal Value for k6.6 Simulation Study6.6.1 Simulating the Stock Price Series6.6.2 Making stockSim() FasterBibliographyFigure 6.1Figure 6.2Figure 6.3Figure 6.4Figure 6.5Figure 6.6Figure 6.7Figure 6.8Figure 6.9
7.1 Introduction7.1.1 The Monte Carlo Method7.1.2 Computational Topics7.2 Exploring the Random Process7.3 Generating Offspring7.3.1 Checking the Results7.3.2 Considering Alternative Implementations7.4 Profiling and Improving Our Code7.5 From One Job's Offspring to an Entire Generation7.6 Unit Testing7.7 A Structure for the Function's Return Value7.8 The Family Tree: Simulating the Branching Process7.9 Replicating the Simulation7.9.1 Analyzing the Simulation Results7.10 ExercisesBibliographyFigure 7.1Figure 7.2Figure 7.3Figure 7.4Figure 7.5Figure 7.6Figure 7.7Figure 7.8Figure 7.9
8.1 Introduction and Motivation8.1.1 Computational Topics8.2 The Model8.2.1 The Order Cars Move8.3 Implementing the BML Model8.3.1 Creating the Initial Grid Configuration8.3.2 Testing the Grid Creation Function8.3.3 Displaying the Grid8.3.4 Visualizing the Grid8.3.5 Simple and Convenient Object-Oriented Programming8.3.6 Moving the Cars8.4 Evaluating the Performance of the Code8.5 Implementing the BML Model in C8.5.1 The Algorithm in C8.5.2 Compiling, Loading, and Calling the C Code8.6 Running the Simulations8.6.1 Exploring Car Velocity8.7 Experimental CompilationBibliographyFigure 8.1Figure 8.2Figure 8.3Figure 8.4Figure 8.5Figure 8.6Figure 8.7Figure 8.8Figure 8.9Figure 8.10Figure 8.11Figure 8.12Figure 8.13
9.1 Introduction9.1.1 Computational Topics9.2 Blackjack Basics9.2.1 Testing Functions9.3 Playing a Hand of Blackjack9.3.1 Creating Functions for the Player's Actions9.4 Strategies for Playing9.4.1 Developing the Optimal Strategy9.5 Playing Many Games9.6 A More Accurate Card Dealer Shoe9.7 Counting Cards9.8 Putting It All Together9.9 ExercisesBibliographyFigure 9.1Figure 9.2Figure 9.3Figure 9.4Table 9.1Table 9.2
10.1 Introduction10.1.1 Computational Topics10.2 Sean Lahman's Database10.2.1 Connecting to the Baseball Database from within R10.3 Aggregating Salaries into Payroll10.4 Merging Payroll Data with Information in Other Tables10.4.1 Adding Team Names to the Payroll Data10.4.2 Adding World Series Records to the Payroll Data10.5 Exploring the Extreme Salaries10.6 ExercisesBibliographyFigure 10.1Figure 10.2Table 10.1
11.1 Introduction11.1.1 Computational Topics11.2 Acquiring the Data11.2.1 Extracting Latitude and Longitude from a CSV File11.3 Integrating Data from Different Sources11.4 Preparing the Data for Plotting11.4.1 Redoing the Merge of the Factbook and Location Data11.5 Plotting with Google Earth™11.6 Extracting Demographic Information from the CIA XML File11.7 Generating KML Directly11.8 Additional Computational Tasks11.8.1 Creating Plotting Symbols11.8.2 Efficiency in Generating KML from Strings11.8.3 Extracting Latitude and Longitude from an HTML File11.9 ExercisesBibliographyFigure 11.1Figure 11.2Figure 11.3Figure 11.4Figure 11.5Figure 11.6Figure 11.7Figure 11.8Figure 11.9Figure 11.10
12.1 Introduction and Motivation12.1.1 Computational Topics12.2 Exploring Different Web Sites12.3 Preliminary/Exploratory Scraping: The Kaggle Job List12.3.1 Processing the Text12.3.2 Generalizing to Other Posts12.3.3 Scraping the Kaggle Post List12.4 Scraping CyberCoders.com12.4.1 Getting the Skill List from a Job Post12.4.2 Finding the Links to Job Postings in the Search Results12.4.3 Finding the Next Page of Job Post Search Results12.4.4 Putting It All Together12.5 A Reusable Generic Framework for Arbitrary Sites12.6 Scraping Career Builder12.7 Scraping Monster.com12.8 Analyzing the Results: The Important Skills12.9 Note on Web Scraping12.10 ExercisesBibliographyFigure 12.1Figure 12.2Figure 12.3Figure 12.4Figure 12.5Figure 12.6Figure 12.7Figure 12.8Figure 12.9Figure 12.10Figure 12.11

Content preview from Data Science in R

Chapter 11

CIA Factbook Mashup

Deborah Nolan

University of California, Berkeley

Duncan Temple Lang

University of California, Davis

11.1 Introduction

The tremendous increase in data that are freely available on the Web has created numerous possibilities for extracting data from different sources, putting them together, and creating exciting new types of visualizations. These visualizations, sometimes called “mashups,” are typically interactive and displayed on the Web. According to Wikipedia [13],

The term [mashup] implies easy, fast integration, frequently using open application programming interfaces (API) and data sources to produce enriched results that were not necessarily the original reason for producing the raw source data. ... The main ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Functional Programming in R: Advanced Statistical Programming for Data Science, Analysis and Finance

Thomas Mailund

Analyzing Baseball Data with R

Max Marchi, Jim Albert

Modeling Techniques in Predictive Analytics with Python and R: A Guide to Data Science

Thomas W. Miller

Graphical Data Analysis with R

Antony Unwin

Publisher Resources

ISBN: 9781482234817