Professional SQL Server® 2008 Integration Services

Chapter 14. Understanding and Tuning the Data Flow Engine

This chapter dives under the hood of SSIS to consider the architecture of the engine and its components, and then best practices for design and optimization including the following concepts:

• Control Flow and Data Flow comparison

• Data Flow Transformation types

• Data Flow buffer architecture and execution trees

• Monitoring Data Flow execution

• Data Flow design practices

• Tuning the Data Flow engine

• Performance monitoring

The initial part of this chapter is more abstract and theoretical, but we'll then move into the practical and tangible. In the concluding sections, you will take the knowledge you have developed here and bring it to application, considering a methodology to optimization and looking at a few real-world scenarios.

The SSIS Engine

Before learning about buffers, asynchronous components, and execution trees, consider this analogy — traffic management. Have you ever driven in a big city and wondered how the traffic system works? It's remarkable to consider how the traffic lights are all coordinated in a city. In Manhattan, for example, a taxi drive can take you from midtown to downtown in minutes — in part because the lights are timed in a rolling fashion to maintain efficiency. The heavy fine assessed to anyone who "locks the box" (remains in the intersection after the light turns red) demonstrates how detrimental it is to interfere with the synchronization of such a complex traffic grid.

Contrast the efficiency of Manhattan with ...