Jobs

Depending upon the SparkContext, the Jobs tab shows the status of all the Spark jobs in a Spark application. When you access the Jobs tab on the Spark UI using a web browser at http://localhost:4040 (for standalone mode), you should observe the following options:

  • User: This shows the active user who has submitted the Spark job
  • Total Uptime: This shows the total uptime for the jobs
  • Scheduling Mode: In most cases, it is first-in-first-out (aka FIFO)
  • Active Jobs: This shows the number of active jobs
  • Completed Jobs: This shows the number of completed jobs
  • Event Timeline: This shows the timeline of a job that has completed its execution

Internally, the Jobs tab is represented by the JobsTab class, which is a custom SparkUI tab with the ...

Get Apache Spark 2: Data Processing and Real-Time Analytics now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.