Big Data Analytics with Spark: A Practitioner’s Guide to Using Spark for Large-Scale Data Processing, Machine Learning, and Graph Analytics, and High-Velocity Data Stream Processing

CHAPTER 5

Writing a Spark Application

This chapter discusses how to write a data processing application in Scala using Spark. Chapter 2 covered the basics of Scala, which you need to know to get started.

The mechanics of writing a Spark application in Scala is not very different from writing a standard Scala application. A Spark application is basically a Scala application that uses Spark as a library. The API provided by the Spark library was discussed in Chapter 3. You will use that API to write a simple data processing application. The goal is to show you how to write, compile, and run a Spark application, so the data processing logic will ...