Patterns for Parallel Software Design

8.2. 8.2 Parallel Software Design

Parallel software design is critical to effective parallel programming, since it is significantly harder than programming sequential programs on single-processor computers. The history of parallel systems [And91] [Bri77] [Fos94] [CSG97] [PB90] [ST96] afford examples in which scientific code (such as numerical analysis, fluid dynamics, etc.) has often been hand-crafted for specific machines and problems, at considerable expense.

Parallel software design begins when a need for high performance is identified, and software designers start creating a parallel software system. Often the hardware and software resources are given. For example, a parallel program might have to be designed using a specific programming language for specific hardware. Such a hardware platform will define other important elements, such as operating system or middleware. The problem of parallelization is normally described in terms of a data set and an algorithm that performs operations on it. This algorithm can be a sequential algorithm or a parallelized algorithm. The main performance goal is usually optimizing execution time [PB90] [Pan96].

Parallel programming relies on the coordination of computing resources so that they work simultaneously and efficiently towards a common objective. Achieving this objective requires significant effort from software designers due to the complexity involved. As parallel programming is intended to improve performance, software designers ...