March 2017
Beginner to intermediate
356 pages
7h 11m
English
Let's look at the steps involved in building a streaming application.
val ssc = new StreamingContext(sc, Seconds(5))
sc = Spark Context referenceSeconds(5) is the batch duration. This can be specified in milliseconds, seconds, or minutes.
It is important to note that in local testing, while specifying the master in the configuration object, do not use local or local[1]. This will mean that only a single thread will be used for running the tasks locally.
If you are using an input stream ...
Read now
Unlock full access