Job Ready Python

Lesson 24Extracting, Transforming, and Loading with ETL Scripting

In this and the next lesson, we will build on everything we've learned throughout this book and create a Python script we can use to automate processes related to extracting data from one source, transforming that data to meet the needs of a data scientist, and load the data into a data source accessible to the data scientist.

LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

• Describe a variety of activities related to ETL.
• Leverage Python to extract, transform, and load data from various sources.
• Use Python to create a script that extracts data from one location, transforms the data as necessary, and loads the transformed data into a separate database.
• Leverage Python and object-oriented programming concepts to create a reusable and compact ETL library.

NOTE This lesson uses files located in the Lesson24 folder of the file that you can download from www.wiley.com/go/jobreadypython.

ETL SCRIPTING IN PYTHON

Data engineers use extract-transform-load (ETL) processes widely in data warehouses to move data between databases, servers, and machines. ETL processes are an intersection of process engineering and technology. It is important to think about ETL processes as actual processes and not as physical implementations of the data. An ETL process can be used in the following situations:

• Access data in a source database or other storage location and load it into a different database ...