How it works...

We started the analysis by using a very simple, yet powerful method of a pandas DataFrame—describe. It printed summary statistics, such as count, mean, min/max, and quartiles of all the numeric variables in the DataFrame. By inspecting these metrics, we could infer the value range of a certain feature, or whether the distribution was skewed (by looking at the difference between mean and median). Also, we could easily spot values outside the plausible range—for example, a negative or very small age.

We can include additional percentiles in the describe method by passing an extra argument percentiles=[.99] —in this case, we added the 99th percentile.

The count metric represents the number of non-null observations, so it is also ...

Get Python for Finance Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.