O'Reilly logo

Fast Data Processing with Spark 2 - Third Edition by Krishna Sankar

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Developing Spark with Eclipse

Eclipse is the IDE of choice for Java developers. The best way is to install the Scala version (2.11.8) of Eclipse from http://scala-ide.org/. This will enable one to develop the Scala and Java programs.

Let's first create a Scala project like so:

Developing Spark with Eclipse

From Spark 2.0.0 onwards, they have changed the packaging, so we have to include spark-2.0.0/assembly/target/scala-2.11/jars in Add External Jars… as shown in the following screenshot:

Developing Spark with Eclipse

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required