TensorFlow Machine Learning Projects

Understanding the Sparkdl interface 

To access Spark functionality in the deep learning pipeline, we need to use a Spark driver program. From Spark 2.0.0, we have a single point entry using SparkSession. The simplest way to do this is by using builder:

SparkSession.builder().getOrCreate()

This can allow us to get an existing session or create a new session. At the time of instantiation, we can use the .config(), .master(), and .appName() methods to set configuration options, set the Spark master, and set the application name.

To read and manipulate images, Sparkdl provides the ImageSchema class. Out of its many methods, we'll be using the readImages method to read the directory of images. It returns a Spark DataFrame with a single column  ...