Skip to Main Content
Big Data Simplified
book

Big Data Simplified

by Sayan Goswami, Amit Kumar Das, Sourabh Mukherjee
June 2019
Beginner to intermediate content levelBeginner to intermediate
360 pages
10h 55m
English
Pearson Education India
Content preview from Big Data Simplified
124 | Big Data Simplied
In the above code, RDD ‘wordPair’ is created from existing RDD ‘word’ using map() transforma-
tion which contains word and its starting character together.
Features of Spark RDD
In-memory Computation: The data inside RDD are stored in memory for as long as you want
to store. Keeping the data in-memory improves the performance by an order of magnitudes.
Lazy Evaluation: The data inside RDDs are not evaluated on the go. The changes or the
computation is performed only after an action is triggered. Thus, it limits how much work it
has to do.
Fault Tolerance: Upon the failure of worker node, using lineage of operations, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Big Data

Big Data

James Warren, Nathan Marz
Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications

Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications

Gary D. Miner, John Elder, Andrew Fast, Thomas Hill, Robert Nisbet, Dursun Delen
Data Wrangling with Python

Data Wrangling with Python

Jacqueline Kazil, Katharine Jarmul

Publisher Resources

ISBN: 9789353941505