Chapter 14. Scalable Algorithms
In this chapter, we discuss the challenges associated with writing efficient and scalable analytics running on Spark. We will start by introducing the reader to the general concepts of distributed parallelization and scalability and how they relate to Spark. We will recap over Spark's distributed architecture giving the reader an understanding of its underlying principles and how this supports the parallel processing paradigm. We will learn about the characteristics of scalable analytics and the elements of Spark that underpin these (for example, RDD, combineByKey, and GraphX).
Next, we will learn about why sometimes even basic algorithms, despite working at small scale, will often fail in big data. We'll see how ...
Get Mastering Spark for Data Science now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
