Beginning Apache Spark 2: With Resilient Distributed Datasets, Spark SQL, Structured Streaming and Spark Machine Learning library

© Hien Luu 2018

Hien LuuBeginning Apache Spark 2https://doi.org/10.1007/978-1-4842-3579-9_2

2. Working with Apache Spark

Hien Luu¹ 

(1)

SAN JOSE, California, USA

 

This chapter provides details about the different ways of working with Spark, including using the Spark shell, submitting a Spark application from the command line, and using a hosted cloud platform called Databricks. The last part of this chapter is geared toward software engineers who want to set up the Apache Spark source code on a local machine to examine the Spark code and learn how certain features were implemented.

Downloading and Installing Spark

For the purposes of learning or experimenting with Spark, it is good to install Spark locally on your computer. This way you can easily try ...