I feel the need, the need for speed.
—Actor Tom Cruise in Top Gun
Ask any given group of IT managers to name the most important aspects of an ETL process, and nine out of ten will put raw speed at the top of the list. Although raw throughput is important in any ETL process, the “need for speed” must be balanced against other requirements such as resource contention with other processes; correct and consistent results; and solution maintainability, manageability, and robustness.
When people hear about tuning the data flow, they often think of increasing raw speed. Processing speed is important, but the speed-only point of view is very one-dimensional. Poor ETL speed is usually the symptom and ...