O'Reilly logo

Mastering Apache Spark by Mike Frampton

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Cluster management

Selecting the Clusters menu option provides a list of your current Databricks clusters and their status. Of course, currently you have none. Selecting the Add Cluster option allows you to create one. Note that the amount of memory you specify determines the size of your cluster. There is a minimum of 54 GB required to create a cluster with a single master and worker. For each additional 54 GB specified, another worker is added.

Cluster management

The following screenshot is a concatenated image, showing a new cluster called semclust1 being created and in a Pending state. While Pending, the cluster has no dashboard, and the cluster nodes are not accessible. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required