O'Reilly logo

Learning pandas - Second Edition by Michael Heydt

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Selecting out or dropping missing data

One technique of handling missing data is to simply remove it from your dataset. A scenario for this would be where data is sampled at regular intervals but devices are offline and hence a reading is not recorded.

The pandas library makes this possible using several techniques. One is through Boolean selection using the results of .isnull() or .notnull() to retrieve the values that are NaN or non NaN out of a Series object. The following example demonstrates selection of all non-NaN values from the c4 column of a DataFrame:

Pandas also provides a convenience function, .dropna(), which drops the items ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required