A single machine is the simplest use case for Spark. It is also a great way to sanity check your build. In
spark/bin, there is a shell script called
run-example, which can be used to launch a Spark job. The
run-example script takes the name of a Spark class and some arguments. Earlier, we used the
run-example script from the
/bin directory to calculate the value of Pi. There is a collection of the sample Spark jobs in
All of the sample programs take the parameter,
master (the cluster manager), which can be the URL of a distributed cluster or
N is the number of threads.
Going back to our
run-example script, it invokes the more general
bin/spark-submit script. For ...