Chapter 3. Building and Running a Spark Application
Using Spark in an interactive mode with the Spark shell has limited permanence and does not work in Java. Building Spark jobs is a bit trickier than building a normal application as all dependencies have to be available on all the machines that are in your cluster. This chapter will cover the process of building a Java and Scala Spark job with Maven or sbt (simple-build-tool) and how to build Spark jobs with a non-Maven aware build system. A reference website to build Spark is http://spark.apache.org/docs/latest/building-spark.html.
Building your Spark project with sbt
The sbt is a popular build tool for Scala that supports building both Scala and Java codes. Building Spark projects with sbt is ...