As we demonstrated in Chapter 5, Machine Learning Workouts on IBM Cloud, there is plenty of essential functionality common to the pandas data structures to support preprocessing and analysis of your data. In this example though, we are going look at examples of data explorations again but this time using Spark DataFrame methods.
For example, earlier we loaded a data file using Insert pandas DataFrame; this time, we can reload that file using the same steps, but this time selecting Insert SparkSession DataFrame. The code generated will include the import ibmos2spark and from pyspark.sql import SparkSession commands and load the data into SparkSession DataFrame (rather than a pandas DataFrame):
import ibmos2spark# @hidden_cell
