Data Lake for Enterprises

Job Manager

The main functions performed by the Job Manager (master node) are as follows:

• Prepares to parallelize execution of tasks.
• Task Scheduling: Schedules the tasks on the task managers. It deploys, stops, and cancels tasks in the task manager.
• Contains the checkpoint coordinator, which is used to trigger checkpoints in the task manager.
• Execution Tracking: Tracks the execution of tasks in the task manager and gathers task results.
• Gathers and keeps the metadata associated with the dataflow graph.

The following figure shows the anatomy of Job Manager in detail and also shows its link to the next component:

Figure 08: Flink Job Manager ...