Hands-On Data Analysis with Pandas

Mitigating the issues

We are in an unsatisfactory state with our data, and while we can work to make it better, the best plan of action isn't always evident. Perhaps the easiest thing we can do when faced with this class of data issues is to remove the duplicate rows. However, it is crucial that we evaluate the ramifications such a decision may have on our analysis. Even in cases where it appears that the data we are working with was collected from a larger dataset that had additional columns, thus making all our data distinct, we can't be sure that the removal of these columns is the reason the remaining data was duplicated—we would need to consult the source of the data and any available documentation.

Since we know that both of the stations ...