Learning Data Mining with Python - Second Edition

Cleaning up the dataset

After looking at the output, we can see a number of problems:

• The date is just a string and not a date object
• From visually inspecting the results, the headings aren't complete or correct

These issues come from the data and we could fix this by altering the data itself. However, in doing this, we could forget the steps we took or misapply them; that is, we can't replicate our results. As with the previous section where we used pipelines to track the transformations we made to a dataset, we will use pandas to apply transformations to the raw data itself.

The pandas.read_csv function has parameters to fix each of these issues, which we can specify when loading the file. We can also change the headings after loading ...