book

Data Science with Python and Dask

Name: Data Science with Python and Dask
Author: Jesse Daniel
ISBN: 9781617295607

by Jesse Daniel

July 2019

Intermediate to advanced

296 pages

9h 1m

English

Manning Publications

Audiobook available

Read now

Unlock full access

Cover
Titlepage
Copyright
contents in brief
contents
Dedication
preface
acknowledgments
about this book
Who should read this bookHow this book is organized: A roadmapAbout the code liveBook discussion forum
about the author

about the cover illustration
Part 1: The building blocks of scalable computing
Chapter 1: Why scalable computing matters
1.1 Why Dask?1.2 Cooking with DAGs1.3 Scaling out, concurrency, and recovery1.3.1 Scaling up vs. scaling out1.3.2 Concurrency and resource management1.3.3 Recovering from failures1.4 Introducing a companion datasetSummary
Chapter 2: Introducing Dask
2.1 Hello Dask: A first look at the DataFrame API2.1.1 Examining the metadata of Dask objects2.1.2 Running computations with the compute method2.1.3 Making complex computations more efficient with persist2.2 Visualizing DAGs2.2.1 Visualizing a simple DAG using Dask Delayed objects2.2.2 Visualizing more complex DAGs with loops and collections2.2.3 Reducing DAG complexity with persist2.3 Task scheduling2.3.1 Lazy computations2.3.2 Data localitySummary
Part 2: Working with structured data using Dask DataFrames
Chapter 3: Introducing Dask DataFrames
3.1 Why use DataFrames?3.2 Dask and Pandas3.2.1 Managing DataFrame partitioning3.2.2 What is the shuffle?3.3 Limitations of Dask DataFramesSummary
Chapter 4: Loading data into DataFrames
4.1 Reading data from text files4.1.1 Using Dask datatypes4.1.2 Creating schemas for Dask DataFrames4.2 Reading data from relational databases4.3 Reading data from HDFS and S34.4 Reading data in Parquet formatSummary
Chapter 5: Cleaning and transforming DataFrames
5.1 Working with indexes and axes5.1.1 Selecting columns from a DataFrame5.1.2 Dropping columns from a DataFrame5.1.3 Renaming columns in a DataFrame5.1.4 Selecting rows from a DataFrame5.2 Dealing with missing values5.2.1 Counting missing values in a DataFrame5.2.2 Dropping columns with missing values5.2.3 Imputing missing values5.2.4 Dropping rows with missing data5.2.5 Imputing multiple columns with missing values5.3 Recoding data5.4 Elementwise operations5.5 Filtering and reindexing DataFrames5.6 Joining and concatenating DataFrames5.6.1 Joining two DataFrames5.6.2 Unioning two DataFrames5.7 Writing data to text files and Parquet files5.7.1 Writing to delimited text files5.7.2 Writing to Parquet filesSummary
Chapter 6: Summarizing and analyzing DataFrames
6.1 Descriptive statistics6.1.1 What are descriptive statistics?6.1.2 Calculating descriptive statistics with Dask6.1.3 Using the describe method for descriptive statistics6.2 Built-In aggregate functions6.2.1 What is correlation?6.2.2 Calculating correlations with Dask DataFrames6.3 Custom aggregate functions6.3.1 Testing categorical variables with the t-test6.3.2 Using custom aggregates to implement the Brown-Forsythe test6.4 Rolling (window) functions6.4.1 Preparing data for a rolling function6.4.2 Using the rolling method to apply a window functionSummary
Chapter 7: Visualizing DataFrames with Seaborn
7.1 The prepare-reduce-collect-plot pattern7.2 Visualizing continuous relationships with scatterplot and regplot7.2.1 Creating a scatterplot with Dask and Seaborn7.2.2 Adding a linear regression line to the scatterplot7.2.3 Adding a nonlinear regression line to a scatterplot7.3 Visualizing categorical relationships with violinplot7.3.1 Creating a violinplot with Dask and Seaborn7.3.2 Randomly sampling data from a Dask DataFrame7.4 Visualizing two categorical relationships with heatmapSummary
Chapter 8: Visualizing location data with Datashader
8.1 What is Datashader and how does it work?8.1.1 The five stages of the Datashader rendering pipeline8.1.2 Creating a Datashader Visualization8.2 Plotting location data as an interactive heatmap8.2.1 Preparing geographic data for map tiling8.2.2 Creating the interactive heatmapSummary
Part 3: Extending and deploying Dask
Chapter 9: Working with Bags and Arrays
9.1 Reading and parsing unstructured data with Bags9.1.1 Selecting and viewing data from a Bag9.1.2 Common parsing issues and how to overcome them9.1.3 Working with delimiters9.2 Transforming, filtering, and folding elements9.2.1 Transforming elements with the map method9.2.2 Filtering Bags with the filter method9.2.3 Calculating descriptive statistics on Bags9.2.4 Creating aggregate functions using the foldby method9.3 Building Arrays and DataFrames from Bags9.4 Using Bags for parallel text analysis with NLTK9.4.1 The basics of bigram analysis9.4.2 Extracting tokens and filtering stopwords9.4.3 Analyzing the bigramsSummary
Chapter 10: Machine learning with Dask-ML
10.1 Building linear models with Dask-ML10.1.1 Preparing the data with binary vectorization10.1.2 Building a logistic regression model with Dask-ML10.2 Evaluating and tuning Dask-ML models10.2.1 Evaluating Dask-ML models with the score method10.2.2 Building a naïve Bayes classifier with Dask-ML10.2.3 Automatically tuning hyperparameters10.3 Persisting Dask-ML modelsSummary
Chapter 11: Scaling and deploying Dask
11.1 Building a Dask cluster on Amazon AWS with Docker 11.1.1 Getting started11.1.2 Creating a security key11.1.3 Creating the ECS cluster11.1.4 Configuring the cluster’s networking11.1.5 Creating a shared data drive in Elastic File System11.1.6 Allocating space for Docker images in Elastic Container Repository11.1.7 Building and deploying images for scheduler, worker, and notebook11.1.8 Connecting to the cluster11.2 Running and monitoring Dask jobs on a cluster11.3 Cleaning up the Dask cluster on AWSSummary
appendix: Software installation
Installing additional packages with AnacondaInstalling packages without AnacondaStarting a Jupyter Notebook serverConfiguring NLTK
Index
List of Figures
List of TablesList of Listings

Content preview from Data Science with Python and Dask

Part 1 The building blocks of scalable computing

This part of the book covers some fundamental concepts in scalable computing to give you a good basis for understanding what makes Dask different and how it works “under the hood.”

In chapter 1, you’ll learn what a directed acyclic graph (DAG)is and why it’s useful for scaling out workloads across many different workers.

Chapter 2 explains how Dask uses DAGs as an abstraction to enable you to analyze huge datasets and take advantage of scalability and parallelism whether you’re running your code on a laptop or a cluster of thousands of machines.

Once you’ve completed part 1, you’ll have a basic understanding of the internals of Dask, and you’ll be ready to get some hands-on experience with ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Productive and Efficient Data Science with Python: With Modularizing, Memory profiles, and Parallel/GPU Processing

Publisher Resources

ISBN: 9781617295607Publisher Support Publisher Website

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Data Science with Python and Dask

by Jesse Daniel

Part 1 The building blocks of scalable computing

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.