Apache Spark Deep Learning Cookbook

How to do it...

The following section walks through the steps to explore and visualize the stock market data.

1. Transform the Date column in the dataframe by removing the timestamp using the following script:

import pyspark.sql.functions as fdf = df.withColumn('date', f.to_date('Date'))

2. Create a for-cycle to add three additional columns to the dataframe. The loop breaks apart the date field into year, month, and day, as seen in the following script:

date_breakdown = ['year', 'month', 'day']for i in enumerate(date_breakdown):    index = i[0]    name = i[1]    df = df.withColumn(name, f.split('date', '-')[index])

3. Save a subset of the Spark dataframe to a pandas dataframe called df_plot using the following script: df_plot = df.select('year', 'Adj ...