July 2006
Intermediate to advanced
378 pages
9h 38m
English
book
Mastering Data Warehouse Aggregates: Solutions for Star Schema Performance
by Christopher Adamson
Content preview from Mastering Data Warehouse Aggregates: Solutions for Star Schema Performance
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
6.4. Dropping and Rebuilding Aggregates
The aggregate load processes described in the previous sections perform incremental processing. By considering only data that is new or changed in the base schema, the volume of data to be processed is limited. An alternative approach is to drop the aggregate tables at the start of each load and rebuild them in their entirety. As discussed in Chapter 5, this approach increases the volume of data processed and does not guarantee the consistency of keys. But it also simplifies the load process, particularly for dimension tables.
6.4.1. Dropping and Rebuilding Aggregate Dimension Tables
In a drop-and-rebuild scenario, the load process for aggregate dimensions is dramatically simplified. All information being added to the dimension table is new; there will be no need to process slowly changing dimensions. Comparisons that are costly in terms of processing time are eliminated, and all aggregate dimension rows are inserts.
For an incremental load of an aggregate dimension table, much of the processing centered on identifying new and changed records. This was done through numerous comparisons of incoming data to the existing aggregate dimension. In Figure 6.3, these comparisons identified new records (step 2.1.2), type 1 changes (step 2.1.3), and type 2 changes (step 2.1.5). These comparisons are costly in terms of computing resources. Each may require a query against the base schema, or the establishment and use of a cache by the ETL tool. When ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.