Skip to Content
Practical Predictive Analytics
book

Practical Predictive Analytics

by Ralph Winters
June 2017
Beginner to intermediate content levelBeginner to intermediate
576 pages
15h 22m
English
Packt Publishing
Content preview from Practical Predictive Analytics

First collect the sample

To illustrate this, let's first take a 50% sample of the stop and frisk dataframe. We also want to make sure that the amount of data we extract can be processed easily by base R, which has a memory limitation that is dependent upon the CPU size.

  • The code below will first extract a 50% sample from Spark and store it in a local R dataframe named dflocal.
  • Then it will run an str() command to verify the rowcount and the metadata:
dflocal = collect(sample(df, F,.50,123)) str(dflocal) 
The output indicates that there are 11,311 rows, which is roughly 50% of the 22,563 rows from the Stop and Frisk data.
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Data Superstream: Analytics Engineering

Data Superstream: Analytics Engineering

Alistair Croll, Anna Filippova, Emilie Schario, Lewis Davies, Jacob Frackson, Benn Stancil, Nick Acosta, Elizabeth Caley
R: Predictive Analysis

R: Predictive Analysis

Tony Fischetti, Eric Mayor, Rui Miguel Forte
Python: Advanced Predictive Analytics

Python: Advanced Predictive Analytics

Ashish Kumar, Joseph Babcock

Publisher Resources

ISBN: 9781785886188Supplemental Content