book

Pandas 1.x Cookbook - Second Edition

by Matthew Harrison, Theodore Petrou

February 2020

Intermediate to advanced

626 pages

12h 17m

English

Packt Publishing

Read now

Unlock full access

Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesRunning a Jupyter NotebookDownload the color imagesConventionsGet in touchReviews
Importing pandasIntroductionThe pandas DataFrameHow it works…DataFrame attributesHow to do it…How it works…There's more...Understanding data typesHow to do it…How it works…There's more…Selecting a columnHow to do it…How it works…There's more…Calling Series methodsHow to do it…How it works…There's more…Series operationsHow to do it…How it works…There's more…Chaining Series methodsHow to do it…How it works…There's more…Renaming column namesHow to do it…How it works…There's more…Creating and deleting columnsHow to do it…How it works…There's more…
IntroductionSelecting multiple DataFrame columnsHow to do it...How it works...There's more...Selecting columns with methodsHow to do it...How it works...There's more...Ordering column namesHow to do it...How it works...There's more...Summarizing a DataFrameHow to do it...How it works...There's more...Chaining DataFrame methodsHow to do it...How it works...There's more...DataFrame operationsHow to do it...How it works...There's more...Comparing missing valuesGetting readyHow to do it...How it works...There's more...Transposing the direction of a DataFrame operationHow to do it...How it works...There's more...Determining college campus diversityHow to do it...How it works...There's more...
IntroductionCreating DataFrames from scratchHow to do it...How it works...There's more...Writing CSVHow to do it...There's more...Reading large CSV filesHow to do it...How it works...There's more...Using Excel filesHow to do it...How it works...There's more...Working with ZIP filesHow to do it...How it works...There's more...Working with databasesHow to do it...How it works...Reading JSONHow to do it...How it works...There's more...Reading HTML tablesHow to do it...How it works...There's more...
IntroductionDeveloping a data analysis routineHow to do it…How it works…There's more…Data dictionariesReducing memory by changing data typesHow to do it…How it works…There's more…Selecting the smallest of the largestHow to do it…How it works…There's more…Selecting the largest of each group by sortingHow to do it…How it works…There's more…Replicating nlargest with sort_valuesHow to do it…How it works…Calculating a trailing stop order priceHow to do it…How it works…There's more…
IntroductionSummary statisticsHow to do it…How it works…There's more…Column typesHow to do it…How it works…There's more…Categorical dataHow to do it…How it works…There's more…Continuous dataHow to do it…How it works…There's more…Comparing continuous values across categoriesHow to do it…How it works…There's more…Comparing two continuous columnsHow to do it…How it works…There's more…Comparing categorical values with categorical valuesHow to do it…How it works…Using the pandas profiling libraryHow to do it…How it works…
IntroductionSelecting Series dataHow to do it…How it works…There's more…Selecting DataFrame rowsHow it works…There's more…Selecting DataFrame rows and columns simultaneouslyHow to do it…How it works…There's more…Selecting data with both integers and labelsHow to do it…How it works…There's more…Slicing lexicographicallyHow to do it…How it works…There's more…
IntroductionCalculating Boolean statisticsHow to do it…How it works…There's more…Constructing multiple Boolean conditionsHow to do it…How it works…There's more…Filtering with Boolean arraysHow to do it…How it works…There's more…Comparing row filtering and index filteringHow to do it…How it works…There's more…Selecting with unique and sorted indexesHow to do it…How it works…There's more…Translating SQL WHERE clausesHow to do it…How it works…There's more…Improving the readability of Boolean indexing with the query methodHow to do it…How it works…There's more…Preserving Series size with the .where methodHow to do it…How it works…There's more…Masking DataFrame rowsHow to do it…How it works…There's more…Selecting with Booleans, integer location, and labelsHow to do it…How it works…
IntroductionExamining the Index objectHow to do it…How it works…There's more…Producing Cartesian productsHow to do it…How it works…There's more…Exploding indexesHow to do it…How it works…There's more…Filling values with unequal indexesHow to do it…How it works…There's more…Adding columns from different DataFramesHow to do it…How it works…There's more…Highlighting the maximum value from each columnHow to do it…How it works…There's more…Replicating idxmax with method chainingHow to do it…How it works…There's more…Finding the most common maximum of columnsHow to do it…How it works…There's more…
IntroductionDefining an aggregationHow to do it…How it works…There's more…Grouping and aggregating with multiple columns and functionsHow to do it…How it works…There's more…Removing the MultiIndex after groupingHow to do it…How it works…There's more…Grouping with a custom aggregation functionHow to do it…How it works…There's more…Customizing aggregating functions with *args and **kwargsHow to do it…How it works…There's more…Examining the groupby objectHow to do it…How it works…There's more…Filtering for states with a minority majorityHow to do it…How it works…There's more…Transforming through a weight loss betHow to do it…How it works…There's more…Calculating weighted mean SAT scores per state with applyHow to do it…How it works…There's more…Grouping by continuous variablesHow to do it…How it works…There's more…Counting the total number of flights between citiesHow to do it…How it works…There's more…Finding the longest streak of on-time flightsHow to do it…How it works…There's more…

IntroductionTidying variable values as column names with stackHow to do it…How it works…There's more…Tidying variable values as column names with meltHow to do it…How it works…There's more…Stacking multiple groups of variables simultaneouslyHow to do it…How it works…There's more…Inverting stacked dataHow to do it…How it works…There's more…Unstacking after a groupby aggregationHow to do it…How it works…There's more…Replicating pivot_table with a groupby aggregationHow to do it…How it works…There's more…Renaming axis levels for easy reshapingHow to do it…How it works…There's more…Tidying when multiple variables are stored as column namesHow to do it…How it works…There's more…Tidying when multiple variables are stored as a single columnHow to do it…How it works…There's more…Tidying when two or more values are stored in the same cellHow to do it...How it works…There's more…Tidying when variables are stored in column names and valuesHow to do it…How it works…There's more…
IntroductionAppending new rows to DataFramesHow to do it…How it works…There's more…Concatenating multiple DataFrames togetherHow to do it…How it works…There's more…Understanding the differences between concat, join, and mergeHow to do it…How it works…There's more…Connecting to SQL databasesHow to do it…How it works…There's more…
IntroductionUnderstanding the difference between Python and pandas date toolsHow to do it…How it works…Slicing time series intelligentlyHow to do it…How it works…There's more…Filtering columns with time dataHow to do it…How it works…There's more…Using methods that only work with a DatetimeIndexHow to do it…How it works…There's more…Counting the number of weekly crimesHow to do it…How it works…There's more…Aggregating weekly crime and traffic accidents separatelyHow to do it…How it works…There's more…Measuring crime by weekday and yearHow to do it…How it works…There's more…Grouping with anonymous functions with a DatetimeIndexHow to do it…How it works…Grouping by a Timestamp and another columnHow to do it…How it works…There's more…
IntroductionGetting started with matplotlibObject-oriented guide to matplotlibHow to do it…How it works…There's more…Visualizing data with matplotlibHow to do it…How it works…There's more…Plotting basics with pandasHow to do it…How it works…There's more…Visualizing the flights datasetHow to do it…How it works…Stacking area charts to discover emerging trendsHow to do it…How it works…Understanding the differences between seaborn and pandasHow to do it…How it works…Multivariate analysis with seaborn GridsHow to do it…How it works…There's more…Uncovering Simpson's Paradox in the diamonds dataset with seabornHow to do it…How it works…There's more…
Code to transform dataHow to do it…How it works…Apply performanceHow to do it…How it works…There's more…Improving apply performance with Dask, Pandarell, Swifter, and moreHow to do it…How it works…Inspecting codeHow to do it…How it works…There's more…Debugging in JupyterHow to do it…How it works…There's more…Managing data integrity with Great ExpectationsHow to do it…How it works…Using pytest with pandasHow to do it…How it works…There's more…Generating tests with HypothesisHow to do it…How it works…

Overview

The 'Pandas 1.x Cookbook' offers a recipe-based guide for mastering the powerful Python library, pandas. You will gain practical knowledge for handling and manipulating data efficiently, from the fundamentals to advanced techniques. The book is an essential resource for exploring and analyzing datasets with pandas.

What this Book will help me do

Understand and apply data exploration techniques in pandas.
Use pandas to manipulate, aggregate, and clean datasets to extract meaningful insights.
Combine pandas with Matplotlib and Seaborn to create effective visualizations.
Perform time series analysis and transform datasets for machine learning.
Implement workflows for handling large-scale data that exceeds your computer's memory.

Author(s)

Matthew Harrison and Theodore Petrou are highly experienced educators and practitioners in data science and Python programming. With their extensive expertise in using pandas, they provide insights through practical exercises and approachable narratives. Their aim is to make complex concepts accessible to learners of varying skill levels.

Who is it for?

This book is ideal for Python programmers, analysts, and data scientists seeking to expand their data handling and analysis capabilities. It caters to both beginners who are new to pandas and those looking to deepen their understanding of its advanced features. If your goal is to explore, clean, and analyze complex datasets efficiently, this book is tailored for you.