Skip to Main Content
Apache Spark Deep Learning Cookbook
book

Apache Spark Deep Learning Cookbook

by Ahmed Sherif, Amrith Ravindra, Michal Malohlava, Adnan Masood
July 2018
Intermediate to advanced content levelIntermediate to advanced
474 pages
13h 37m
English
Packt Publishing
Content preview from Apache Spark Deep Learning Cookbook

How it works...

The following section explains the techniques used and insights gained from exploratory data analysis.

  1. The date column in the dataframe is more of a date-time column with the time values all ending in 00:00:00. This is unnecessary for what we will need during our modeling and therefore can be removed from the dataset. Luckily for us, PySpark has a to_date function that can do this quite easily. The dataframe, df, is transformed using the withColumn() function and now only shows the date column without the timestamp, as seen in the following screenshot:
  1. For analysis purposes, we want to extract the day, month, and year from ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Apache Spark for Data Science Cookbook

Apache Spark for Data Science Cookbook

Padma Priya Chitturi
Learning Apache Spark 2

Learning Apache Spark 2

Muhammad Asif Abbasi

Publisher Resources

ISBN: 9781788474221Supplemental Content