O'Reilly logo

Learning Real-time Processing with Spark Streaming by Sumit Gupta

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Troubleshooting

In this section we will talk about tips and tricks which are helpful when solving the most common errors encountered while working with Spark.

Configuring port numbers

Spark binds various network ports for communication and exposing information to developers and administrators. There may be instances where the default ports used by Spark may not be available or may be blocked by the network firewall which in turn will result in modifying the default Spark ports for master/worker or driver.

Here is the list of all ports utilized by Spark and their associated parameters, which need to be configured for any changes http://spark.apache.org/docs/latest/security.html#configuring-ports-for-network-security.

Classpath issues – class not found ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required