O'Reilly logo

Practical Predictive Analytics by Ralph Winters

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

About Spark

At the time of writing, Spark is probably the most popular very large dataset architecture for predictive analytics. Spark is a distributed architecture which helps you manage your large data and makes it easier to analyze. Spark is built upon Hadoop and they share the same filesystem.

However, Spark is not based upon the MapReduce paradigm, and uses the resilient distributed dataset (RDD) structure in order to implement in-memory analytics and manage the parallel processing cluster across all of the nodes of the environment. What that means for analysts is that queries can be very quick, since data is retrieved from memory, which offers much quicker retrieval than disk access. Quicker access means more time for analysis, and ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required