O'Reilly logo

Practical Predictive Analytics by Ralph Winters

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Reading the data from the global views

We saved the diabetes dataset as a table and a global view in the previous section, so we can now read it back using SQL. We will first read the training table (temp.df_view) and then follow by reading the test table (test_view). The resulting objects are both Spark dataframes.

After reading the table in, print the count of the rows. If you want to see the counts as one result, you can use a function such as cat() to print the results on one line:

df = sql("select age,                     pregnant,                     glucose,                     pressure,                     insulin,                     pedigree,                     triceps,                     mass from global_temp.df_view")  test = sql("select age ,                     pregnant,                     glucose,                     pressure,                     insulin,                     pedigree,                     triceps,                     mass from global_temp.test_view")  cat(count(df),count(test)) ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required