Skip to Content
Jupyter Cookbook
book

Jupyter Cookbook

by Toomey, Nikhil Borkar, Nikhil Akki, Juan Tomás Oliva Ramos
April 2018
Beginner content levelBeginner
238 pages
7h 13m
English
Packt Publishing
Content preview from Jupyter Cookbook

How it works...

We have a standard preamble to the coding. All Spark programs need a context to work with. The context is used to define the number of threads and the like. We are only using the defaults. It's important to note that Spark will automatically utilize underlying multiple CPUs and the like as needed, without specific intervention.

Then we load the text file into memory. This is a standard method available in Spark. If we were accessing a database, we might be able to use parallel operations to read different segments of the primary key to split up the file access.

Once the file is loaded, we split each line into words and use a lambda function to tick off each occurrence of a word. The code is truly creating a new record for ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Python Cookbook, 3rd Edition

Python Cookbook, 3rd Edition

David Beazley, Brian K. Jones
Pandas 1.x Cookbook - Second Edition

Pandas 1.x Cookbook - Second Edition

Matthew Harrison, Theodore Petrou
bash Cookbook, 2nd Edition

bash Cookbook, 2nd Edition

Carl Albing, JP Vossen

Publisher Resources

ISBN: 9781788839440Supplemental Content