Spark Streaming, ML, and Windowing Operations

Lesson Objectives

By the end of this lesson, you will be able to:

  • Use the Spark machine learning library
  • Build a collaborative filtering model for movie recommendations
  • Build a system that suggest movies in real time by using Spark streams and machine learning
  • Apply windowing operations to live streams of data

This chapter concludes this book by describing how we can use the streaming feature in collaboration with the machine learning functionality using MLib library.

Introduction

In the last three lessons, we learned about the most relevant concepts regarding Spark and Spark Streaming. We performed practical exercises to learn how to use RDD and the SQL APIs, and we also learned how to ...

Get Big Data Processing with Apache Spark now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.