Professional SQL Server® 2008 Integration Services
by Brian Knight, Erik Veerman, Grant Dickinson, Douglas Hinson, Darren Herbold
Chapter 14. Understanding and Tuning the Data Flow Engine
This chapter dives under the hood of SSIS to consider the architecture of the engine and its components, and then best practices for design and optimization including the following concepts:
Control Flow and Data Flow comparison
Data Flow Transformation types
Data Flow buffer architecture and execution trees
Monitoring Data Flow execution
Data Flow design practices
Tuning the Data Flow engine
Performance monitoring
The initial part of this chapter is more abstract and theoretical, but we'll then move into the practical and tangible. In the concluding sections, you will take the knowledge you have developed here and bring it to application, considering a methodology to optimization and looking at a few real-world scenarios.
The SSIS Engine
Before learning about buffers, asynchronous components, and execution trees, consider this analogy — traffic management. Have you ever driven in a big city and wondered how the traffic system works? It's remarkable to consider how the traffic lights are all coordinated in a city. In Manhattan, for example, a taxi drive can take you from midtown to downtown in minutes — in part because the lights are timed in a rolling fashion to maintain efficiency. The heavy fine assessed to anyone who "locks the box" (remains in the intersection after the light turns red) demonstrates how detrimental it is to interfere with the synchronization of such a complex traffic grid.
Contrast the efficiency of Manhattan with ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access