The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data

Chapter 10. Responsibilities

In this chapter, we discuss managing the development and administration of a successful ETL system. We could have put this chapter at the beginning of the book, before the myriad responsibilities of the ETL system were discussed thoroughly, but we think by putting it at the end of the book, you will better be able to visualize how to manage a team effectively.

The first part of this chapter looks at planning and leadership issues, and the second part descends into more detail of managing the ETL system. Many of these perspectives were developed in Data Warehouse Lifecycle Toolkit.

Note

PROCESS CHECK Planning & Design:

Requirements/Realities → Architecture → Implementation → Release to Ops

Data Flow: Extract → Clean → Conform → Deliver

Planning and Leadership

In some ways, the data warehouse and ETL process are just like any other software development project. When a data warehouse team is established, it usually requires three specialists. The following list contains common roles required to initiate a data warehouse project. The list includes the primary role and the secondary role (in parentheses) that the same individual can perform on small teams.

• Data Modeler (Project Manager). The data modeler must be specially trained in dimensional data modeling and educated in the principles of dimensional modeling.

• ETL Architect/Programmer (DW Architect). The ETL programmer is usually a SQL and database expert as well as an architect. This person establishes the technical ...