O'Reilly logo

Learning Real-time Processing with Spark Streaming by Sumit Gupta

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Your first Spark Streaming program

In this section we will code and deploy our first Spark Streaming job in Scala and Java.

Our streaming job will connect to a specific port number and will receive or capture data at regular intervals. It will evaluate and calculate the count of distinct words in the data received. Lastly, it will print this on the console.

The Spark Streaming example shown in subsequent sections is comprised of two distinct parts:

  • Spark Streaming job: The Spark Streaming job, which contains the actual application logic and will be executed on the Spark cluster
  • Client application: The client application which opens a specific port and will write some data to that port at regular intervals

Let's move forward and see the code for the ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required