book

Python Feature Engineering Cookbook - Second Edition

Name: Python Feature Engineering Cookbook - Second Edition
Author: Soledad Galli
ISBN: 9781804611302

by Soledad Galli

October 2022

Beginner to intermediate

386 pages

8h 20m

English

Packt Publishing

Read now

Unlock full access

Python Feature Engineering Cookbook
ContributorsAbout the authorAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedSectionsGetting readyHow to do it…How it works…There’s more…See alsoGet in touchReviewsShare Your ThoughtsDownload a Free PDF copy of this book
Chapter 1: Imputing Missing Data
Technical requirementsRemoving observations with missing dataHow to do it...How it works...Performing mean or median imputationHow to do it...How it works...Imputing categorical variablesHow to do it...How it works...Replacing missing values with an arbitrary numberHow to do it...How it works...Finding extreme values for imputationHow to do it...How it works...Marking imputed valuesHow to do it...How it works...Performing multivariate imputation by chained equationsHow to do it...How it works...See alsoEstimating missing data with nearest neighborsHow to do it...How it works...
Chapter 2: Encoding Categorical Variables
Technical requirementsCreating binary variables through one-hot encodingHow to do it...How it works...There’s more...Performing one-hot encoding of frequent categoriesHow to do it...How it works...There’s more...Replacing categories with counts or the frequency of observationsHow to do it...How it works...Replacing categories with ordinal numbersHow to do it...How it works...There’s more...Performing ordinal encoding based on the target valueHow to do it...How it works...See alsoImplementing target mean encodingHow to do it...How it works…There’s more…Encoding with the Weight of EvidenceHow to do it...How it works...See alsoGrouping rare or infrequent categoriesHow to do it...How it works...Performing binary encodingHow to do it...How it works...See also
Chapter 3: Transforming Numerical Variables
Transforming variables with the logarithm functionGetting readyHow to do it...How it works...There’s more…Transforming variables with the reciprocal functionHow to do it...How it works...Using the square root to transform variablesHow to do it...How it works…Using power transformationsHow to do it...How it works...Performing Box-Cox transformationHow to do it...How it works...There’s more…Performing Yeo-Johnson transformationHow to do it...How it works...There’s more…
Chapter 4: Performing Variable Discretization
Technical requirementsPerforming equal-width discretizationHow to do it...How it works…See alsoImplementing equal-frequency discretizationHow to do it...How it works…Discretizing the variable into arbitrary intervalsHow to do it...How it works...Performing discretization with k-means clusteringHow to do it...How it works...See alsoImplementing feature binarizationGetting readyHow to do it...How it works…Using decision trees for discretizationHow to do it...How it works...There’s more...
Chapter 5: Working with Outliers
Technical requirementsVisualizing outliers with boxplotsHow to do it...How it works…Finding outliers using the mean and standard deviationHow to do it...How it works…Finding outliers with the interquartile range proximity ruleHow to do it...How it works…Removing outliersHow to do it...How it works...Capping or censoring outliersHow to do it...How it works...There’s more...Capping outliers using quantilesHow to do it...How it works...
Chapter 6: Extracting Features from Date and Time Variables
Technical requirementsExtracting features from dates with pandasGetting readyHow to do it...How it works...There’s more…See alsoExtracting features from time with pandasGetting readyHow to do it...How it works...There’s more…Capturing the elapsed time between datetime variablesHow to do it...How it works...See alsoWorking with time in different time zonesHow to do it...How it works...See alsoAutomating feature extraction with Feature-engineHow to do it...How it works...
Chapter 7: Performing Feature Scaling
Technical requirementsStandardizing the featuresHow to do it...How it works...Scaling to the maximum and minimum valuesHow to do it...How it works...Scaling with the median and quantilesHow to do it...How it works...Performing mean normalizationHow to do it...How it works…There’s more...Implementing maximum absolute scalingGetting readyHow to do it...How it works...There’s more...Scaling to vector unit lengthHow to do it...How it works...
Chapter 8: Creating New Features
Technical requirementsCombining features with mathematical functionsGetting readyHow to do it...How it works...See alsoComparing features to reference variablesHow to do it…How it works...See alsoPerforming polynomial expansionGetting readyHow to do it...How it works...There’s more...See alsoCombining features with decision treesGetting readyHow to do it...How it works...Creating periodic features from cyclical variablesGetting readyHow to do it…How it works…See alsoCreating spline featuresGetting readyHow to do it…How it works…See also

Chapter 9: Extracting Features from Relational Data with Featuretools
Technical requirementsSetting up an entity set and creating features automaticallyGetting readyHow to do it...How it works...See alsoCreating features with general and cumulative operationsGetting readyHow to do it...How it works...Combining numerical featuresHow to do it...How it works...Extracting features from date and timeHow to do it...How it works...There’s more...Extracting features from textGetting readyHow to do it...How it works...Creating features with aggregation primitivesGetting readyHow to do it...How it works...
Chapter 10: Creating Features from a Time Series with tsfresh
Technical requirementsExtracting features automatically from a time seriesGetting readyHow to do it...How it works...See alsoCreating and selecting features for a time seriesHow to do it...How it works...See alsoTailoring feature creation to different time seriesHow to do it...How it works...Creating pre-selected featuresHow to do it...How it works...Embedding feature creation in a scikit-learn pipelineHow to do it...How it works...See also
Chapter 11: Extracting Features from Text Variables
Technical requirementsCounting characters, words, and vocabularyGetting readyHow to do it...How it works...There’s more...See alsoEstimating text complexity by counting sentencesGetting readyHow to do it...How it works...There’s more...Creating features with bag-of-words and n-gramsGetting readyHow to do it...How it works...See alsoImplementing term frequency-inverse document frequencyGetting readyHow to do it...How it works...See alsoCleaning and stemming text variablesGetting readyHow to do it...How it works...
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a Free PDF copy of this book

Content preview from Python Feature Engineering Cookbook - Second Edition

3 Transforming Numerical Variables

Statistical methods used in data analysis make certain assumptions about the data. For example, in the general linear model, it is assumed that the values of the dependent variable (the target) are independent, that there is a linear relationship between the target and the independent (predictor) variables, and that the residuals – that is, the difference between the predictions and the real values of the target – are normally distributed and centered at 0. When these assumptions are not met, the resulting probabilistic statements might not be accurate. To correct for failure in the assumptions and thus improve the performance of the models, we can transform variables before the analysis.

Variable transformation ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781804611302

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Python Feature Engineering Cookbook - Second Edition

by Soledad Galli

3

Transforming Numerical Variables

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.