May 2007
Intermediate to advanced
432 pages
11h 43m
English
Content preview from Expert SQL Server™ 2005 Integration Services
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
12.2. Data Flow Optimization
Integration Services is a platform, meaning that its purpose is to support applications built on its features. The data flow, for example, has no value in and of itself; the value is gained when it is applied to a data-processing purpose (such as warehouse ETL, system integration processes, or DBA data management operations). What this means when it comes to optimization techniques for the data flow is that there are no guaranteed formulas that yield performance gains, because every data flow design has a unique scenario. This section includes principles and common areas for you to evaluate against your data flows to determine applicability.
12.2.1. Pipeline Architecture Review
The SSIS data flow engine uses data buffers to manage data as it flows through the pipeline. As data is extracted form the sources, it is consumed into reserved memory spaces, called buffers. These buffers are acted upon by the various transformations as the data flows through the data flow to the destination. A complete review of the SSIS pipeline engine is found in Chapter 10 of the Professional SQL Server 2005 Integration Services book (Indianapolis: Wiley, 2006). Here are key points to remember:
When data is loaded into a buffer, the transformation logic is applied to the data in place (where possible). In a sense, the transformations pass over the buffers, giving SSIS the capability to stream data through the pipeline. These inline buffer changes are cheap as compared ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.