Scala and Spark for Big Data Analytics

Creating StreamingContext

A new StreamingContext can be created in two ways:

1. Create a StreamingContext using an existing SparkContext as follows:

      StreamingContext(sparkContext: SparkContext, batchDuration: Duration)      scala> val ssc = new StreamingContext(sc, Seconds(10))

2. Create a StreamingContext by providing the configuration necessary for a new SparkContext as follows:

      StreamingContext(conf: SparkConf, batchDuration: Duration)      scala> val conf = new SparkConf().setMaster("local[1]")                                       .setAppName("TextStreams")      scala> val ssc = new StreamingContext(conf, Seconds(10))

3. A third method is to use getOrCreate(), which is used to either recreate a StreamingContext from checkpoint data or to create a new StreamingContext. If checkpoint data ...