The Spark package comes with examples. So, to test whether all required dependencies are working fine, execute the following commands:
cd spark-2.1.0-bin-hadoop2.7
./bin/run-example SparkPi 10
The output of this will be:
Pi is roughly 3.1415431415431416
Before starting spark-shell, we need to setup Hadoop on the local machine. Download Hadoop 2.7.4 version from the following link: http://www-eu.apache.org/dist/hadoop/common/hadoop-2.7.4/hadoop-2.7.4.tar.gz. Untar the files and folders extracted using the following command:
tar -xvf hadoop-2.7.4.tar.gz
The output is as shown in the following screenshot:
Copy the configuration ...
