Jobs

Depending upon the SparkContext, the Jobs tab shows the status of all the Spark jobs in a Spark application. When you access the Jobs tab on the Spark UI using a web browser at http://localhost:4040 (for standalone mode), you should observe the following options:

  • User: This shows the active user who has submitted the Spark job
  • Total Uptime: This shows the total uptime for the jobs
  • Scheduling Mode: In most cases, it is first-in-first-out (aka FIFO)
  • Active Jobs: This shows the number of active jobs
  • Completed Jobs: This shows the number of completed jobs
  • Event Timeline: This shows the timeline of a job that has completed its execution

Internally, the Jobs tab is represented by the JobsTab class, which is a custom SparkUI tab with the ...

Get Scala and Spark for Big Data Analytics now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.