Iterative Design Optimization

At design time, it’s very common to define execution trees to match requirements and the designer’s thought process. Upon careful review, you can identify unnecessary branches and steps to reduce complexity and improve performance.

The following example shows a data flow with a separate execution tree for erroneous records. Keep in mind that an error can be defined simply as a value that doesn’t meet certain criteria. The 10 million rows flowing through the Lookup Error Output pipeline don’t meet the same criteria as the 40 million rows flowing through the main pipeline and are corrected by an expression in the Derived Column transformation. These rows flow down the error pipeline and are then joined back to the main ...

Get Microsoft® SQL Server™ 2005 Integration Services Step by Step now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.