book

Python Feature Engineering Cookbook - Second Edition

Name: Python Feature Engineering Cookbook - Second Edition
Author: Soledad Galli
ISBN: 9781804611302

by Soledad Galli

October 2022

Beginner to intermediate

386 pages

8h 20m

English

Packt Publishing

Read now

Unlock full access

Python Feature Engineering Cookbook
ContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedSectionsGetting readyHow to do it…How it works…There’s more…See alsoGet in touchReviewsShare Your ThoughtsDownload a Free PDF copy of this book
Chapter 1: Imputing Missing Data
Technical requirementsRemoving observations with missing dataHow to do it...How it works...Performing mean or median imputationHow to do it...How it works...Imputing categorical variablesHow to do it...How it works...Replacing missing values with an arbitrary numberHow to do it...How it works...Finding extreme values for imputationHow to do it...How it works...Marking imputed valuesHow to do it...How it works...Performing multivariate imputation by chained equationsHow to do it...How it works...See alsoEstimating missing data with nearest neighborsHow to do it...How it works...
Chapter 2: Encoding Categorical Variables
Technical requirementsCreating binary variables through one-hot encodingHow to do it...How it works...There’s more...Performing one-hot encoding of frequent categoriesHow to do it...How it works...There’s more...Replacing categories with counts or the frequency of observationsHow to do it...How it works...Replacing categories with ordinal numbersHow to do it...How it works...There’s more...Performing ordinal encoding based on the target valueHow to do it...How it works...See alsoImplementing target mean encodingHow to do it...How it works…There’s more…Encoding with the Weight of EvidenceHow to do it...How it works...See alsoGrouping rare or infrequent categoriesHow to do it...How it works...Performing binary encodingHow to do it...How it works...See also
Chapter 3: Transforming Numerical Variables
Transforming variables with the logarithm functionGetting readyHow to do it...How it works...There’s more…Transforming variables with the reciprocal functionHow to do it...How it works...Using the square root to transform variablesHow to do it...How it works…Using power transformationsHow to do it...How it works...Performing Box-Cox transformationHow to do it...How it works...There’s more…Performing Yeo-Johnson transformationHow to do it...How it works...There’s more…
Chapter 4: Performing Variable Discretization
Technical requirementsPerforming equal-width discretizationHow to do it...How it works…See alsoImplementing equal-frequency discretizationHow to do it...How it works…Discretizing the variable into arbitrary intervalsHow to do it...How it works...Performing discretization with k-means clusteringHow to do it...How it works...See alsoImplementing feature binarizationGetting readyHow to do it...How it works…Using decision trees for discretizationHow to do it...How it works...There’s more...
Chapter 5: Working with Outliers
Technical requirementsVisualizing outliers with boxplotsHow to do it...How it works…Finding outliers using the mean and standard deviationHow to do it...How it works…Finding outliers with the interquartile range proximity ruleHow to do it...How it works…Removing outliersHow to do it...How it works...Capping or censoring outliersHow to do it...How it works...There’s more...Capping outliers using quantilesHow to do it...How it works...
Chapter 6: Extracting Features from Date and Time Variables
Technical requirementsExtracting features from dates with pandasGetting readyHow to do it...How it works...There’s more…See alsoExtracting features from time with pandasGetting readyHow to do it...How it works...There’s more…Capturing the elapsed time between datetime variablesHow to do it...How it works...See alsoWorking with time in different time zonesHow to do it...How it works...See alsoAutomating feature extraction with Feature-engineHow to do it...How it works...
Chapter 7: Performing Feature Scaling
Technical requirementsStandardizing the featuresHow to do it...How it works...Scaling to the maximum and minimum valuesHow to do it...How it works...Scaling with the median and quantilesHow to do it...How it works...Performing mean normalizationHow to do it...How it works…There’s more...Implementing maximum absolute scalingGetting readyHow to do it...How it works...There’s more...Scaling to vector unit lengthHow to do it...How it works...
Chapter 8: Creating New Features
Technical requirementsCombining features with mathematical functionsGetting readyHow to do it...How it works...See alsoComparing features to reference variablesHow to do it…How it works...See alsoPerforming polynomial expansionGetting readyHow to do it...How it works...There’s more...See alsoCombining features with decision treesGetting readyHow to do it...How it works...Creating periodic features from cyclical variablesGetting readyHow to do it…How it works…See alsoCreating spline featuresGetting readyHow to do it…How it works…See also

Chapter 9: Extracting Features from Relational Data with Featuretools
Technical requirementsSetting up an entity set and creating features automaticallyGetting readyHow to do it...How it works...See alsoCreating features with general and cumulative operationsGetting readyHow to do it...How it works...Combining numerical featuresHow to do it...How it works...Extracting features from date and timeHow to do it...How it works...There’s more...Extracting features from textGetting readyHow to do it...How it works...Creating features with aggregation primitivesGetting readyHow to do it...How it works...
Chapter 10: Creating Features from a Time Series with tsfresh
Technical requirementsExtracting features automatically from a time seriesGetting readyHow to do it...How it works...See alsoCreating and selecting features for a time seriesHow to do it...How it works...See alsoTailoring feature creation to different time seriesHow to do it...How it works...Creating pre-selected featuresHow to do it...How it works...Embedding feature creation in a scikit-learn pipelineHow to do it...How it works...See also
Chapter 11: Extracting Features from Text Variables
Technical requirementsCounting characters, words, and vocabularyGetting readyHow to do it...How it works...There’s more...See alsoEstimating text complexity by counting sentencesGetting readyHow to do it...How it works...There’s more...Creating features with bag-of-words and n-gramsGetting readyHow to do it...How it works...See alsoImplementing term frequency-inverse document frequencyGetting readyHow to do it...How it works...See alsoCleaning and stemming text variablesGetting readyHow to do it...How it works...
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a Free PDF copy of this book

Content preview from Python Feature Engineering Cookbook - Second Edition

1 Imputing Missing Data

Missing data, that is, the absence of values for certain observations, is an unavoidable problem in most data sources. Scikit-learn, the most commonly used Python library for machine learning, does not support missing values as input to machine learning models. Thus, we must remove observations with missing data or transform them into permitted values.

The act of replacing missing data with statistical estimates of missing values is called imputation. The goal of any imputation technique is to produce a complete dataset. There are multiple imputation methods that we can use, depending on whether the data is missing at random, the proportion of missing values, and the machine learning model we intend to use. In this chapter, ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781804611302

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Python Feature Engineering Cookbook - Second Edition

by Soledad Galli

1

Imputing Missing Data

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.