5.1. The ETL Process5.2. Extracting Data from the Operational Systems5.2.1. Identifying Data That Has Changed5.2.2. Oracle Change Data CapturePublishing Change DataSynchronous CDCAsynchronous CDCSubscribing to Change DataCreating a SubscriptionProcessing the Change DataStep 1: Extend the windowStep 2: Select Data from the Subscriber ViewStep 3: Purge the WindowEnding the SubscriptionTransporting the Changes to the Staging Area5.3. Transforming the Data into a Common Representation5.3.1. Integrating Data from Multiple Sources5.3.2. Cleansing Data5.3.3. Deriving New Data5.3.4. Generating Warehouse Keys5.3.5. Choosing the Optimal Place to Perform the Transformations5.4. Loading the Warehouse5.4.1. Using SQL*Loader to Load the WarehouseUsing Oracle Enterprise Manager Load WizardThe Control File5.4.2. The Data FileSQL*Loader Modes of OperationData Load OptionsScheduling the Load OperationMonitoring Progress of the Load OperationInspecting the SQL*Loader LogOptimizing SQL*Loader PerformanceSQL*Loader Direct Path Load of a Single PartitionStep 1: Create a TablespaceStep 2: Add a PartitionStep 3: Disable All Referential Integrity Constraints and TriggersStep 4: Load the DataStep 5: Inspect the LogStep 6: Reenable All Constraints and Triggers, Rebuild IndexesSQL*Loader Parallel Direct Path LoadStep 1: Disable All Constraints and TriggersStep 2: Drop all IndexesStep 3: Load the DataStep 4: Inspect the LogStep 5: Reenable All Constraints and Triggers, Recreate All IndexesTransformations Using SQL*LoaderSQL*Loader Postload OperationsStep 1: Inspect the LogsStep 2: Process the Load ExceptionsStep 3: Reenable Data Integrity ConstraintsStep 4: Handle Constraint ViolationsStep 5: Enabling Constraints without ValidationCheck for Unusable IndexesRebuild unusable indexes5.4.3. Loading the Warehouse Using Data PumpData Pump Import/Export—impdp and expdpSpecifying the Location of the Datafile and Log Files for Data Pump ToolsMoving Data between DatabasesImproved Job Monitoring and Control5.4.4. Loading the Warehouse Using External TablesCreating an External TableAccessing Data Stored in an External TableLoading Data From an External TableLoading Data in Parallel Using External TablesUsing Data Pump External Tables to Move and Load Data5.4.5. Loading the Warehouse Using Transportable TablespacesStep 1: Create a Tablespace in the OLTP SystemStep 2: Move the Data for April 2004 into a Table in the Newly Created TablespaceStep 3: Alter the Tablespace So That It Is Read-OnlyStep 4: EXPORT the MetadataStep 5: Convert the Datafiles (Optional)Step 6: Transport the TablespaceStep 7: Import the MetadataStep 8: Alter the Tablespace to Read/Write5.4.6. Loading the Dimensions Using SQL MERGE5.5. Transformations inside the Oracle Database5.5.1. Transformations That Cleanse Data and Derive New DataProcessing With More Power: The REGEXP FunctionsRegular Expression Basics and SearchingRegular Expressions and SubstringsRegular Expressions to Manipulate Data5.5.2. Validating Data Using a Dimension5.5.3. Looking up the Warehouse Key5.5.4. Table Functions5.5.5. Transformations That Split One Data Source into Multiple Targets5.5.6. Moving Data from a Staging Table into the Fact TableStep 1: Create a New Tablespace for the Jan Purchases and the Jan Purchases IndexStep 2: Add a Partition to the Purchases TableStep 3: Move the table into the new partitionMoving Data Using Exchange PartitionMoving Data Between Tables Using Direct Path InsertCreating a New Table Using Create Table As Select5.6. Postload Operations5.6.1. Step 1: Gather Optimizer Statistics for the Tables5.6.2. Step 2: Verify the Dimensions5.6.3. Step 3: Refresh the materialized views5.6.4. Step 4: Gather Optimizer Statistics for the Materialized Views5.6.5. Step 5: Back up the Database Table, or Partition5.6.6. Step 6: Publish the Data5.7. Using Tools for the ETL process5.8. Summary