book

Data Science with Python and Dask

by Jesse Daniel

July 2019

Intermediate to advanced

296 pages

9h 1m

English

Manning Publications

Read now

Unlock full access

Who should read this bookHow this book is organized: A roadmapAbout the code liveBook discussion forum

1.1 Why Dask?1.2 Cooking with DAGs1.3 Scaling out, concurrency, and recovery1.3.1 Scaling up vs. scaling out1.3.2 Concurrency and resource management1.3.3 Recovering from failures1.4 Introducing a companion datasetSummary
2.1 Hello Dask: A first look at the DataFrame API2.1.1 Examining the metadata of Dask objects2.1.2 Running computations with the compute method2.1.3 Making complex computations more efficient with persist2.2 Visualizing DAGs2.2.1 Visualizing a simple DAG using Dask Delayed objects2.2.2 Visualizing more complex DAGs with loops and collections2.2.3 Reducing DAG complexity with persist2.3 Task scheduling2.3.1 Lazy computations2.3.2 Data localitySummary
3.1 Why use DataFrames?3.2 Dask and Pandas3.2.1 Managing DataFrame partitioning3.2.2 What is the shuffle?3.3 Limitations of Dask DataFramesSummary
4.1 Reading data from text files4.1.1 Using Dask datatypes4.1.2 Creating schemas for Dask DataFrames4.2 Reading data from relational databases4.3 Reading data from HDFS and S34.4 Reading data in Parquet formatSummary
5.1 Working with indexes and axes5.1.1 Selecting columns from a DataFrame5.1.2 Dropping columns from a DataFrame5.1.3 Renaming columns in a DataFrame5.1.4 Selecting rows from a DataFrame5.2 Dealing with missing values5.2.1 Counting missing values in a DataFrame5.2.2 Dropping columns with missing values5.2.3 Imputing missing values5.2.4 Dropping rows with missing data5.2.5 Imputing multiple columns with missing values5.3 Recoding data5.4 Elementwise operations5.5 Filtering and reindexing DataFrames5.6 Joining and concatenating DataFrames5.6.1 Joining two DataFrames5.6.2 Unioning two DataFrames5.7 Writing data to text files and Parquet files5.7.1 Writing to delimited text files5.7.2 Writing to Parquet filesSummary
6.1 Descriptive statistics6.1.1 What are descriptive statistics?6.1.2 Calculating descriptive statistics with Dask6.1.3 Using the describe method for descriptive statistics6.2 Built-In aggregate functions6.2.1 What is correlation?6.2.2 Calculating correlations with Dask DataFrames6.3 Custom aggregate functions6.3.1 Testing categorical variables with the t-test6.3.2 Using custom aggregates to implement the Brown-Forsythe test6.4 Rolling (window) functions6.4.1 Preparing data for a rolling function6.4.2 Using the rolling method to apply a window functionSummary
7.1 The prepare-reduce-collect-plot pattern7.2 Visualizing continuous relationships with scatterplot and regplot7.2.1 Creating a scatterplot with Dask and Seaborn7.2.2 Adding a linear regression line to the scatterplot7.2.3 Adding a nonlinear regression line to a scatterplot7.3 Visualizing categorical relationships with violinplot7.3.1 Creating a violinplot with Dask and Seaborn7.3.2 Randomly sampling data from a Dask DataFrame7.4 Visualizing two categorical relationships with heatmapSummary
8.1 What is Datashader and how does it work?8.1.1 The five stages of the Datashader rendering pipeline8.1.2 Creating a Datashader Visualization8.2 Plotting location data as an interactive heatmap8.2.1 Preparing geographic data for map tiling8.2.2 Creating the interactive heatmapSummary
9.1 Reading and parsing unstructured data with Bags9.1.1 Selecting and viewing data from a Bag9.1.2 Common parsing issues and how to overcome them9.1.3 Working with delimiters9.2 Transforming, filtering, and folding elements9.2.1 Transforming elements with the map method9.2.2 Filtering Bags with the filter method9.2.3 Calculating descriptive statistics on Bags9.2.4 Creating aggregate functions using the foldby method9.3 Building Arrays and DataFrames from Bags9.4 Using Bags for parallel text analysis with NLTK9.4.1 The basics of bigram analysis9.4.2 Extracting tokens and filtering stopwords9.4.3 Analyzing the bigramsSummary
10.1 Building linear models with Dask-ML10.1.1 Preparing the data with binary vectorization10.1.2 Building a logistic regression model with Dask-ML10.2 Evaluating and tuning Dask-ML models10.2.1 Evaluating Dask-ML models with the score method10.2.2 Building a naïve Bayes classifier with Dask-ML10.2.3 Automatically tuning hyperparameters10.3 Persisting Dask-ML modelsSummary
11.1 Building a Dask cluster on Amazon AWS with Docker 11.1.1 Getting started11.1.2 Creating a security key11.1.3 Creating the ECS cluster11.1.4 Configuring the cluster’s networking11.1.5 Creating a shared data drive in Elastic File System11.1.6 Allocating space for Docker images in Elastic Container Repository11.1.7 Building and deploying images for scheduler, worker, and notebook11.1.8 Connecting to the cluster11.2 Running and monitoring Dask jobs on a cluster11.3 Cleaning up the Dask cluster on AWSSummary
Installing additional packages with AnacondaInstalling packages without AnacondaStarting a Jupyter Notebook serverConfiguring NLTK
List of TablesList of Listings

Content preview from Data Science with Python and Dask

8 Visualizing location data with Datashader

This chapter covers

Using Datashader to visualize many datapoints when downsampling isn’t appropriate
Plotting interactive heatmaps using Datashader and Bokeh

In the previous chapter, we looked at a few ways to gain insights from data using visualization. However, every method we looked at relied on finding workarounds to reduce the size of the data we used for plotting. Whether by randomly sampling, filtering, or aggregating the data, we used these downsampling techniques to overcome the inherent limitations of Seaborn and Matplotlib. Although we’ve shown that these techniques can be useful, downsampling can cause us to miss patterns in the data because we’re throwing data away. This issue becomes ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Start your free trial

Publisher Resources

ISBN: 9781617295607Other Supplemental Content Publisher Support Publisher Website Supplemental Content Purchase Link

Data Science with Python and Dask

by Jesse Daniel

8 Visualizing location data with Datashader

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

You might also like

Practical Data Science with Python

Python: End-to-end Data Analysis

Python for Data Science

Machine Learning with Python Cookbook

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,and much more.

You might also like

Practical Data Science with Python

Python: End-to-end Data Analysis

Python for Data Science

Machine Learning with Python Cookbook

Publisher Resources

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.