O'Reilly logo

Practical Predictive Analytics by Ralph Winters

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Reading the entire file into memory

Now that we have some aggregate counts, we have some idea of what we will be looking at. So we will read the entire file into a Spark dataframe via SQL and print the first record (you can print more if you like).

Just by looking at the first record(s), you can see that there are a mix of binary yes/no flags and some numeric variables:

df <- sql("SELECT * FROM stopfrisk") head(df,1) 

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required