book

Pandas 1.x Cookbook - Second Edition

Name: Pandas 1.x Cookbook - Second Edition
ISBN: 9781839213106

by Matthew Harrison, Theodore Petrou

February 2020

Intermediate to advanced

626 pages

12h 17m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesRunning a Jupyter NotebookDownload the color imagesConventionsGet in touchReviews
Pandas Foundations
Importing pandasIntroductionThe pandas DataFrameHow it works…DataFrame attributesHow to do it…How it works…There's more...Understanding data typesHow to do it…How it works…There's more…Selecting a columnHow to do it…How it works…There's more…Calling Series methodsHow to do it…How it works…There's more…Series operationsHow to do it…How it works…There's more…Chaining Series methodsHow to do it…How it works…There's more…Renaming column namesHow to do it…How it works…There's more…Creating and deleting columnsHow to do it…How it works…There's more…
Essential DataFrame Operations
IntroductionSelecting multiple DataFrame columnsHow to do it...How it works...There's more...Selecting columns with methodsHow to do it...How it works...There's more...Ordering column namesHow to do it...How it works...There's more...Summarizing a DataFrameHow to do it...How it works...There's more...Chaining DataFrame methodsHow to do it...How it works...There's more...DataFrame operationsHow to do it...How it works...There's more...Comparing missing valuesGetting readyHow to do it...How it works...There's more...Transposing the direction of a DataFrame operationHow to do it...How it works...There's more...Determining college campus diversityHow to do it...How it works...There's more...
Creating and Persisting DataFrames
IntroductionCreating DataFrames from scratchHow to do it...How it works...There's more...Writing CSVHow to do it...There's more...Reading large CSV filesHow to do it...How it works...There's more...Using Excel filesHow to do it...How it works...There's more...Working with ZIP filesHow to do it...How it works...There's more...Working with databasesHow to do it...How it works...Reading JSONHow to do it...How it works...There's more...Reading HTML tablesHow to do it...How it works...There's more...
Beginning Data Analysis
IntroductionDeveloping a data analysis routineHow to do it…How it works…There's more…Data dictionariesReducing memory by changing data typesHow to do it…How it works…There's more…Selecting the smallest of the largestHow to do it…How it works…There's more…Selecting the largest of each group by sortingHow to do it…How it works…There's more…Replicating nlargest with sort_valuesHow to do it…How it works…Calculating a trailing stop order priceHow to do it…How it works…There's more…
Exploratory Data Analysis
IntroductionSummary statisticsHow to do it…How it works…There's more…Column typesHow to do it…How it works…There's more…Categorical dataHow to do it…How it works…There's more…Continuous dataHow to do it…How it works…There's more…Comparing continuous values across categoriesHow to do it…How it works…There's more…Comparing two continuous columnsHow to do it…How it works…There's more…Comparing categorical values with categorical valuesHow to do it…How it works…Using the pandas profiling libraryHow to do it…How it works…
Selecting Subsets of Data
IntroductionSelecting Series dataHow to do it…How it works…There's more…Selecting DataFrame rowsHow it works…There's more…Selecting DataFrame rows and columns simultaneouslyHow to do it…How it works…There's more…Selecting data with both integers and labelsHow to do it…How it works…There's more…Slicing lexicographicallyHow to do it…How it works…There's more…
Filtering Rows
IntroductionCalculating Boolean statisticsHow to do it…How it works…There's more…Constructing multiple Boolean conditionsHow to do it…How it works…There's more…Filtering with Boolean arraysHow to do it…How it works…There's more…Comparing row filtering and index filteringHow to do it…How it works…There's more…Selecting with unique and sorted indexesHow to do it…How it works…There's more…Translating SQL WHERE clausesHow to do it…How it works…There's more…Improving the readability of Boolean indexing with the query methodHow to do it…How it works…There's more…Preserving Series size with the .where methodHow to do it…How it works…There's more…Masking DataFrame rowsHow to do it…How it works…There's more…Selecting with Booleans, integer location, and labelsHow to do it…How it works…
Index Alignment
IntroductionExamining the Index objectHow to do it…How it works…There's more…Producing Cartesian productsHow to do it…How it works…There's more…Exploding indexesHow to do it…How it works…There's more…Filling values with unequal indexesHow to do it…How it works…There's more…Adding columns from different DataFramesHow to do it…How it works…There's more…Highlighting the maximum value from each columnHow to do it…How it works…There's more…Replicating idxmax with method chainingHow to do it…How it works…There's more…Finding the most common maximum of columnsHow to do it…How it works…There's more…
Grouping for Aggregation, Filtration, and Transformation
IntroductionDefining an aggregationHow to do it…How it works…There's more…Grouping and aggregating with multiple columns and functionsHow to do it…How it works…There's more…Removing the MultiIndex after groupingHow to do it…How it works…There's more…Grouping with a custom aggregation functionHow to do it…How it works…There's more…Customizing aggregating functions with *args and **kwargsHow to do it…How it works…There's more…Examining the groupby objectHow to do it…How it works…There's more…Filtering for states with a minority majorityHow to do it…How it works…There's more…Transforming through a weight loss betHow to do it…How it works…There's more…Calculating weighted mean SAT scores per state with applyHow to do it…How it works…There's more…Grouping by continuous variablesHow to do it…How it works…There's more…Counting the total number of flights between citiesHow to do it…How it works…There's more…Finding the longest streak of on-time flightsHow to do it…How it works…There's more…

Restructuring Data into a Tidy Form
IntroductionTidying variable values as column names with stackHow to do it…How it works…There's more…Tidying variable values as column names with meltHow to do it…How it works…There's more…Stacking multiple groups of variables simultaneouslyHow to do it…How it works…There's more…Inverting stacked dataHow to do it…How it works…There's more…Unstacking after a groupby aggregationHow to do it…How it works…There's more…Replicating pivot_table with a groupby aggregationHow to do it…How it works…There's more…Renaming axis levels for easy reshapingHow to do it…How it works…There's more…Tidying when multiple variables are stored as column namesHow to do it…How it works…There's more…Tidying when multiple variables are stored as a single columnHow to do it…How it works…There's more…Tidying when two or more values are stored in the same cellHow to do it...How it works…There's more…Tidying when variables are stored in column names and valuesHow to do it…How it works…There's more…
Combining Pandas Objects
IntroductionAppending new rows to DataFramesHow to do it…How it works…There's more…Concatenating multiple DataFrames togetherHow to do it…How it works…There's more…Understanding the differences between concat, join, and mergeHow to do it…How it works…There's more…Connecting to SQL databasesHow to do it…How it works…There's more…
Time Series Analysis
IntroductionUnderstanding the difference between Python and pandas date toolsHow to do it…How it works…Slicing time series intelligentlyHow to do it…How it works…There's more…Filtering columns with time dataHow to do it…How it works…There's more…Using methods that only work with a DatetimeIndexHow to do it…How it works…There's more…Counting the number of weekly crimesHow to do it…How it works…There's more…Aggregating weekly crime and traffic accidents separatelyHow to do it…How it works…There's more…Measuring crime by weekday and yearHow to do it…How it works…There's more…Grouping with anonymous functions with a DatetimeIndexHow to do it…How it works…Grouping by a Timestamp and another columnHow to do it…How it works…There's more…
Visualization with Matplotlib, Pandas, and Seaborn
IntroductionGetting started with matplotlibObject-oriented guide to matplotlibHow to do it…How it works…There's more…Visualizing data with matplotlibHow to do it…How it works…There's more…Plotting basics with pandasHow to do it…How it works…There's more…Visualizing the flights datasetHow to do it…How it works…Stacking area charts to discover emerging trendsHow to do it…How it works…Understanding the differences between seaborn and pandasHow to do it…How it works…Multivariate analysis with seaborn GridsHow to do it…How it works…There's more…Uncovering Simpson's Paradox in the diamonds dataset with seabornHow to do it…How it works…There's more…
Debugging and Testing Pandas
Code to transform dataHow to do it…How it works…Apply performanceHow to do it…How it works…There's more…Improving apply performance with Dask, Pandarell, Swifter, and moreHow to do it…How it works…Inspecting codeHow to do it…How it works…There's more…Debugging in JupyterHow to do it…How it works…There's more…Managing data integrity with Great ExpectationsHow to do it…How it works…Using pytest with pandasHow to do it…How it works…There's more…Generating tests with HypothesisHow to do it…How it works…
Other Books You May Enjoy
Index

Overview

The 'Pandas 1.x Cookbook' offers a recipe-based guide for mastering the powerful Python library, pandas. You will gain practical knowledge for handling and manipulating data efficiently, from the fundamentals to advanced techniques. The book is an essential resource for exploring and analyzing datasets with pandas.

What this Book will help me do

Understand and apply data exploration techniques in pandas.
Use pandas to manipulate, aggregate, and clean datasets to extract meaningful insights.
Combine pandas with Matplotlib and Seaborn to create effective visualizations.
Perform time series analysis and transform datasets for machine learning.
Implement workflows for handling large-scale data that exceeds your computer's memory.

Author(s)

Matthew Harrison and Theodore Petrou are highly experienced educators and practitioners in data science and Python programming. With their extensive expertise in using pandas, they provide insights through practical exercises and approachable narratives. Their aim is to make complex concepts accessible to learners of varying skill levels.

Who is it for?

This book is ideal for Python programmers, analysts, and data scientists seeking to expand their data handling and analysis capabilities. It caters to both beginners who are new to pandas and those looking to deepen their understanding of its advanced features. If your goal is to explore, clean, and analyze complex datasets efficiently, this book is tailored for you.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781839213106

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills