Using Spark Notebooks for quick iteration of ideas

In this section, we will answer the following questions:

  • What are Spark Notebooks?
  • How do you start Spark Notebooks?
  • How do you use Spark Notebooks?

Let's start with setting up a Jupyter Notebook-like environment for Spark. Spark Notebook is just an interactive and reactive data science environment that uses Scala and Spark.

If we view the GitHub page (https://github.com/spark-notebook/spark-notebook), we can see that what the Notebooks do is actually very straightforward, as shown in the following screenshot:

If we look at a Spark Notebook, we can see that they look very much like what ...

Get Hands-On Big Data Analytics with PySpark now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.