Design – Build – Run: Applied Practices and Principles for Production-Ready Software Development

16.5. Summary

Using the approaches outlined in this chapter, you can come up with a generic and extensible batch architecture and framework. You can reduce the number of moving parts to support the functional batch processes by re-using common components and various configuration settings. You can also use generic processes during development and testing, as well as in the production environment.

The following are the key points to take away from this chapter:

• Have a batch framework. Implementing a reusable framework will reduce development and test effort, as well as reduce the number of moving parts in the solution. The framework should encapsulate the execution and control to avoid all jobs having to implement this functionality.

• Keep each job simple and discrete. This is the first rule of thumb for batch. The scheduler is designed for managing dependencies, ordering, and job execution, so the jobs themselves shouldn't include logic for checking or determining whether other jobs have executed (either correctly or incorrectly). The job wouldn't be executed by the scheduler if the relevant dependencies hadn't been met, so the job should simply focus on the discrete processing it needs to perform.

• Ensure batch processing is as quick as possible. The batch window is ever decreasing, so each batch process should perform its processing as quickly and efficiently as possible. Where possible (and appropriate) the batch jobs should make use of bulk processing to improve performance.