on-demand course

Scaling Python for Big Data

with O'Reilly Media, Inc.

December 2016

Intermediate

English

Closed Captioning available in German, English, Spanish, French, Japanese, Korean, Portuguese (Portugal, Brazil), Chinese (Simplified), Chinese (Traditional)

Watch now

Unlock full access

Includes

Earns Badge

Course outline

Welcome To The Course
2m 53s
About The Author
1m 55s
Introduction To Automation
2m 48s
Adventures With Servers
6m 37s
Being A Good Systems Caretaker
6m 3s
What Is A Queue?
2m 32s
What Is A Consumer? What Is A Producer?
2m 0s
Why Celery?
1m 49s
Celery Architecture & Set Up
5m 25s
Writing Your First Tasks
7m 49s
Deploying Your Tasks
6m 8s
Scaling Your Workers
8m 52s
Monitoring With Flower
5m 5s
Advanced Celery Features
6m 0s
Why Dask?
3m 1s
First Steps With Dask
10m 8s
Dask Bags
10m 18s
Dask Distributed
9m 58s
What Are Data Pipelines? What Is Dag?
2m 37s
Luigi And Airflow: A Comparison
5m 50s
First Steps With Luigi
7m 12s
More Complex Luigi Tasks
9m 17s
Introduction To Hadoop
8m 21s
First Steps With Airflow
8m 7s
Custom Tasks With Airflow
9m 16s
Advanced Airflow: Subdags And Branches
11m 17s
Using Luigi With Hadoop
10m 15s
Apache Spark
8m 28s
Apache Spark Streaming
6m 32s
Django Channels
9m 39s
And Many More
5m 59s
Introduction To Testing With Python
7m 24s
Property-Based Testing With Hypothesis
6m 9s
What's Next?
3m 57s
Introduction And Course Overview
2m 1s
About The Author
1m 2s
Installing Python
4m 38s
Installing iPython And Using Notebooks
6m 28s
Download And Setup
3m 24s
Running The Spark Shell
5m 35s
Running The Spark Shell With iPython
6m 38s
What Is A Resilient Distributed Dataset - RDD?
4m 54s
Reading A Text File
3m 34s
Actions
2m 13s
Transformations
2m 30s
Persisting Data
4m 11s
Map
3m 4s
Filter
3m 56s
Flatmap
3m 16s
MapPartitions
4m 7s
MapPartitionsWithIndex
1m 51s
Sample
2m 36s
Union
1m 11s
Intersection
1m 28s
Distinct
2m 2s
Cartesian
3m 17s
Pipe
3m 40s
Coalesce
2m 12s
Repartition
2m 29s
RepartitionAndSortWithinPartitions
3m 58s
Reduce
4m 19s
Collect
1m 56s
Count
3m 5s
First
1m 20s
Take
1m 5s
TakeSample
3m 3s
TakeOrdered
2m 10s
SaveAsTextFile
4m 9s
CountByKey
2m 40s
ForEach
3m 11s
GroupByKey
2m 31s
ReduceByKey
3m 30s
AggregateByKey
3m 44s
SortByKey
2m 47s
Join
4m 16s
CoGroup
2m 9s
WholeTextFile
3m 15s
Pickle Files
3m 59s
HadoopInputFormat
5m 35s
HadoopOutputFormat
5m 31s
Broadcast Variables
4m 17s
Accumulators
5m 8s
Using A Custom Accumulator
4m 52s
Partitioning
7m 56s
Spark Standalone Cluster
4m 26s
Mesos
3m 38s
Yarn
2m 28s
Client Versus Cluster Mode
2m 41s
Spark Streaming
4m 21s
Dataframes And SQL
3m 28s
MLlib
4m 29s
Resources And Where To Go From Here
1m 2s
Wrap Up
1m 28s

Overview

If you have some Python experience, and you want to take it to the next level, this practical, hands-on course will be a helpful resource. Video tutorials in this course will show you how to use Python for distributed task processing, and perform large-scale data processing in Spark using the PySpark API.