book

Data Science with Python and Dask

by Jesse Daniel

July 2019

Intermediate to advanced

296 pages

9h 1m

English

Manning Publications

Read now

Unlock full access

Who should read this bookHow this book is organized: A roadmapAbout the code liveBook discussion forum

1.1 Why Dask?1.2 Cooking with DAGs1.3 Scaling out, concurrency, and recovery1.3.1 Scaling up vs. scaling out1.3.2 Concurrency and resource management1.3.3 Recovering from failures1.4 Introducing a companion datasetSummary
2.1 Hello Dask: A first look at the DataFrame API2.1.1 Examining the metadata of Dask objects2.1.2 Running computations with the compute method2.1.3 Making complex computations more efficient with persist2.2 Visualizing DAGs2.2.1 Visualizing a simple DAG using Dask Delayed objects2.2.2 Visualizing more complex DAGs with loops and collections2.2.3 Reducing DAG complexity with persist2.3 Task scheduling2.3.1 Lazy computations2.3.2 Data localitySummary
3.1 Why use DataFrames?3.2 Dask and Pandas3.2.1 Managing DataFrame partitioning3.2.2 What is the shuffle?3.3 Limitations of Dask DataFramesSummary
4.1 Reading data from text files4.1.1 Using Dask datatypes4.1.2 Creating schemas for Dask DataFrames4.2 Reading data from relational databases4.3 Reading data from HDFS and S34.4 Reading data in Parquet formatSummary
5.1 Working with indexes and axes5.1.1 Selecting columns from a DataFrame5.1.2 Dropping columns from a DataFrame5.1.3 Renaming columns in a DataFrame5.1.4 Selecting rows from a DataFrame5.2 Dealing with missing values5.2.1 Counting missing values in a DataFrame5.2.2 Dropping columns with missing values5.2.3 Imputing missing values5.2.4 Dropping rows with missing data5.2.5 Imputing multiple columns with missing values5.3 Recoding data5.4 Elementwise operations5.5 Filtering and reindexing DataFrames5.6 Joining and concatenating DataFrames5.6.1 Joining two DataFrames5.6.2 Unioning two DataFrames5.7 Writing data to text files and Parquet files5.7.1 Writing to delimited text files5.7.2 Writing to Parquet filesSummary
6.1 Descriptive statistics6.1.1 What are descriptive statistics?6.1.2 Calculating descriptive statistics with Dask6.1.3 Using the describe method for descriptive statistics6.2 Built-In aggregate functions6.2.1 What is correlation?6.2.2 Calculating correlations with Dask DataFrames6.3 Custom aggregate functions6.3.1 Testing categorical variables with the t-test6.3.2 Using custom aggregates to implement the Brown-Forsythe test6.4 Rolling (window) functions6.4.1 Preparing data for a rolling function6.4.2 Using the rolling method to apply a window functionSummary
7.1 The prepare-reduce-collect-plot pattern7.2 Visualizing continuous relationships with scatterplot and regplot7.2.1 Creating a scatterplot with Dask and Seaborn7.2.2 Adding a linear regression line to the scatterplot7.2.3 Adding a nonlinear regression line to a scatterplot7.3 Visualizing categorical relationships with violinplot7.3.1 Creating a violinplot with Dask and Seaborn7.3.2 Randomly sampling data from a Dask DataFrame7.4 Visualizing two categorical relationships with heatmapSummary
8.1 What is Datashader and how does it work?8.1.1 The five stages of the Datashader rendering pipeline8.1.2 Creating a Datashader Visualization8.2 Plotting location data as an interactive heatmap8.2.1 Preparing geographic data for map tiling8.2.2 Creating the interactive heatmapSummary
9.1 Reading and parsing unstructured data with Bags9.1.1 Selecting and viewing data from a Bag9.1.2 Common parsing issues and how to overcome them9.1.3 Working with delimiters9.2 Transforming, filtering, and folding elements9.2.1 Transforming elements with the map method9.2.2 Filtering Bags with the filter method9.2.3 Calculating descriptive statistics on Bags9.2.4 Creating aggregate functions using the foldby method9.3 Building Arrays and DataFrames from Bags9.4 Using Bags for parallel text analysis with NLTK9.4.1 The basics of bigram analysis9.4.2 Extracting tokens and filtering stopwords9.4.3 Analyzing the bigramsSummary
10.1 Building linear models with Dask-ML10.1.1 Preparing the data with binary vectorization10.1.2 Building a logistic regression model with Dask-ML10.2 Evaluating and tuning Dask-ML models10.2.1 Evaluating Dask-ML models with the score method10.2.2 Building a naïve Bayes classifier with Dask-ML10.2.3 Automatically tuning hyperparameters10.3 Persisting Dask-ML modelsSummary
11.1 Building a Dask cluster on Amazon AWS with Docker 11.1.1 Getting started11.1.2 Creating a security key11.1.3 Creating the ECS cluster11.1.4 Configuring the cluster’s networking11.1.5 Creating a shared data drive in Elastic File System11.1.6 Allocating space for Docker images in Elastic Container Repository11.1.7 Building and deploying images for scheduler, worker, and notebook11.1.8 Connecting to the cluster11.2 Running and monitoring Dask jobs on a cluster11.3 Cleaning up the Dask cluster on AWSSummary
Installing additional packages with AnacondaInstalling packages without AnacondaStarting a Jupyter Notebook serverConfiguring NLTK
List of TablesList of Listings

Overview

Dask is a native parallel analytics tool designed to integrate seamlessly with the libraries you’re already using, including Pandas, NumPy, and Scikit-Learn. With Dask you can crunch and work with huge datasets, using the tools you already have. And Data Science with Python and Dask is your guide to using Dask for your data projects without changing the way you work!

About the Technology

An efficient data pipeline means everything for the success of a data science project. Dask is a flexible library for parallel computing in Python that makes it easy to build intuitive workflows for ingesting and analyzing large, distributed datasets. Dask provides dynamic task scheduling and parallel collections that extend the functionality of NumPy, Pandas, and Scikit-learn, enabling users to scale their code from a single laptop to a cluster of hundreds of machines with ease.

About the Book

Data Science with Python and Dask teaches you to build scalable projects that can handle massive datasets. After meeting the Dask framework, you’ll analyze data in the NYC Parking Ticket database and use DataFrames to streamline your process. Then, you’ll create machine learning models using Dask-ML, build interactive visualizations, and build clusters using AWS and Docker.

What's Inside

Working with large, structured and unstructured datasets
Visualization with Seaborn and Datashader
Implementing your own algorithms
Building distributed apps with Dask Distributed
Packaging and deploying Dask apps

About the Reader

For data scientists and developers with experience using Python and the PyData stack.

About the Author

Jesse Daniel is an experienced Python developer. He taught Python for Data Science at the University of Denver and leads a team of data scientists at a Denver-based media technology company.

We interviewed Jesse as a part of our Six Questions series. Check it out here.

Quotes
The most comprehensive coverage of Dask to date, with real-world examples that made a difference in my daily work.
- Al Krinker, United States Patent and Trademark Office

An excellent alternative to PySpark for those who are not on a cloud platform. The author introduces Dask in a way that speaks directly to an analyst.
- Jeremy Loscheider, Panera Bread

A greatly paced introduction to Dask with real-world datasets.
- George Thomas, R&D Architecture Manhattan Associates

The ultimate resource to quickly get up and running with Dask and parallel processing in Python.
- Gustavo Patino, Oakland University William Beaumont School of Medicine