Apache Spark 2.x for Java Developers

Initializing SparkSession

Let's create a SparkSession object. SparkSession follows the builder design pattern, therefore we can initialize SparkSession in the following way:

SparkSession sparkSession =SparkSession.builder() 
.master("local") 
.appName("Spark Session Example") 
.getOrCreate(); 

You must have noticed that we have not created any SparkContext or SparkConf objects for initializing SparkSession. The SparkConf and SparkContext are encapsulated in SparkSession. No explicit initialization of these objects is required with SparkSession.

The SparkConf and SparkContext can be accessed using the SparkSession object as follows:

 SparkContext sparkContext = sparkSession.sparkContext(); SparkConf conf = sparkSession.sparkContext().getConf(); ...