Job Ready Python

Lesson 25Improving ETL Scripting

From Lesson 24, we have a basic ETL script that can be used to extract data from a variety of sources, perform basic transformations on the extracted data, and load the transformed data into a variety of target datasets. In this lesson, we will tweak the existing script to make it more efficient and easier to use.

LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

• Set up Python classes to use static methods.
• Add exception handling to the existing extract class.
• Update the extract class to allow a developer to add custom extract methods.

CONVERTING TO STATIC METHODS FOR THE EXTRACT CLASS

Static methods, much like class methods, are methods that are bound to a class rather than its object. They do not require a class instance creation, so they are not dependent on the state of the object. To make the methods in our classes more flexible, we will add @staticmethod to each method in our classes.

Update the extract class to include @staticmethod with each method in the class, as shown in Listing 25.1. You should also remove the self argument from each method because it is not needed when we use static methods.

LISTING 25.1Converting classes to static in the extract class

class extract:
    @staticmethod
    def fromCSV(file_path, delimiter = ",", quotechar = "|"):
        if not file_path:
            raise Exception("You must provide a valid file path.")
        import csv
        dataset = list()
        with open(file_path) as f: 
 csv_file = csv.DictReader(f, ...