book

Data Science with Python and Dask

by Jesse Daniel

July 2019

Intermediate to advanced

296 pages

9h 1m

English

Manning Publications

Read now

Unlock full access

Who should read this bookHow this book is organized: A roadmapAbout the code liveBook discussion forum

1.1 Why Dask?1.2 Cooking with DAGs1.3 Scaling out, concurrency, and recovery1.3.1 Scaling up vs. scaling out1.3.2 Concurrency and resource management1.3.3 Recovering from failures1.4 Introducing a companion datasetSummary
2.1 Hello Dask: A first look at the DataFrame API2.1.1 Examining the metadata of Dask objects2.1.2 Running computations with the compute method2.1.3 Making complex computations more efficient with persist2.2 Visualizing DAGs2.2.1 Visualizing a simple DAG using Dask Delayed objects2.2.2 Visualizing more complex DAGs with loops and collections2.2.3 Reducing DAG complexity with persist2.3 Task scheduling2.3.1 Lazy computations2.3.2 Data localitySummary
3.1 Why use DataFrames?3.2 Dask and Pandas3.2.1 Managing DataFrame partitioning3.2.2 What is the shuffle?3.3 Limitations of Dask DataFramesSummary
4.1 Reading data from text files4.1.1 Using Dask datatypes4.1.2 Creating schemas for Dask DataFrames4.2 Reading data from relational databases4.3 Reading data from HDFS and S34.4 Reading data in Parquet formatSummary
5.1 Working with indexes and axes5.1.1 Selecting columns from a DataFrame5.1.2 Dropping columns from a DataFrame5.1.3 Renaming columns in a DataFrame5.1.4 Selecting rows from a DataFrame5.2 Dealing with missing values5.2.1 Counting missing values in a DataFrame5.2.2 Dropping columns with missing values5.2.3 Imputing missing values5.2.4 Dropping rows with missing data5.2.5 Imputing multiple columns with missing values5.3 Recoding data5.4 Elementwise operations5.5 Filtering and reindexing DataFrames5.6 Joining and concatenating DataFrames5.6.1 Joining two DataFrames5.6.2 Unioning two DataFrames5.7 Writing data to text files and Parquet files5.7.1 Writing to delimited text files5.7.2 Writing to Parquet filesSummary
6.1 Descriptive statistics6.1.1 What are descriptive statistics?6.1.2 Calculating descriptive statistics with Dask6.1.3 Using the describe method for descriptive statistics6.2 Built-In aggregate functions6.2.1 What is correlation?6.2.2 Calculating correlations with Dask DataFrames6.3 Custom aggregate functions6.3.1 Testing categorical variables with the t-test6.3.2 Using custom aggregates to implement the Brown-Forsythe test6.4 Rolling (window) functions6.4.1 Preparing data for a rolling function6.4.2 Using the rolling method to apply a window functionSummary
7.1 The prepare-reduce-collect-plot pattern7.2 Visualizing continuous relationships with scatterplot and regplot7.2.1 Creating a scatterplot with Dask and Seaborn7.2.2 Adding a linear regression line to the scatterplot7.2.3 Adding a nonlinear regression line to a scatterplot7.3 Visualizing categorical relationships with violinplot7.3.1 Creating a violinplot with Dask and Seaborn7.3.2 Randomly sampling data from a Dask DataFrame7.4 Visualizing two categorical relationships with heatmapSummary
8.1 What is Datashader and how does it work?8.1.1 The five stages of the Datashader rendering pipeline8.1.2 Creating a Datashader Visualization8.2 Plotting location data as an interactive heatmap8.2.1 Preparing geographic data for map tiling8.2.2 Creating the interactive heatmapSummary
9.1 Reading and parsing unstructured data with Bags9.1.1 Selecting and viewing data from a Bag9.1.2 Common parsing issues and how to overcome them9.1.3 Working with delimiters9.2 Transforming, filtering, and folding elements9.2.1 Transforming elements with the map method9.2.2 Filtering Bags with the filter method9.2.3 Calculating descriptive statistics on Bags9.2.4 Creating aggregate functions using the foldby method9.3 Building Arrays and DataFrames from Bags9.4 Using Bags for parallel text analysis with NLTK9.4.1 The basics of bigram analysis9.4.2 Extracting tokens and filtering stopwords9.4.3 Analyzing the bigramsSummary
10.1 Building linear models with Dask-ML10.1.1 Preparing the data with binary vectorization10.1.2 Building a logistic regression model with Dask-ML10.2 Evaluating and tuning Dask-ML models10.2.1 Evaluating Dask-ML models with the score method10.2.2 Building a naïve Bayes classifier with Dask-ML10.2.3 Automatically tuning hyperparameters10.3 Persisting Dask-ML modelsSummary
11.1 Building a Dask cluster on Amazon AWS with Docker 11.1.1 Getting started11.1.2 Creating a security key11.1.3 Creating the ECS cluster11.1.4 Configuring the cluster’s networking11.1.5 Creating a shared data drive in Elastic File System11.1.6 Allocating space for Docker images in Elastic Container Repository11.1.7 Building and deploying images for scheduler, worker, and notebook11.1.8 Connecting to the cluster11.2 Running and monitoring Dask jobs on a cluster11.3 Cleaning up the Dask cluster on AWSSummary
Installing additional packages with AnacondaInstalling packages without AnacondaStarting a Jupyter Notebook serverConfiguring NLTK
List of TablesList of Listings

Content preview from Data Science with Python and Dask

about this book

Who should read this book

Data Science with Python and Dask takes you on a hands-on journey through a typical data science workflow—from data cleaning through deployment—using Dask. The book begins by presenting some foundational knowledge of scalable computing and explains how Dask takes advantage of those concepts to operate on datasets big and small. Building on that foundation, it then turns its focus to preparing, analyzing, visualizing, and modeling various real-world datasets to give you tangible examples of how to use Dask to perform common data science tasks. Finally, the book ends with a step-by-step walkthrough of deploying your very own Dask cluster on AWS to scale out your analysis code.

Data Science with Python ...