Data Science on AWS

Chapter 11. Streaming Analytics and Machine Learning

In the previous chapters, we assumed that we have all of our data available in a centralized static location, such as our S3-based data lake. Real-world data is continuously streaming from many different sources across the world simultaneously. We need to perform machine learning on streams of data for use cases such as fraud prevention and anomaly detection where the latency of batch processing is not acceptable. We may also want to run continuous analytics on real-time data streams to gain competitive advantage and shorten the time to business insights.

In this chapter, we move from our customer reviews training dataset into a real-world scenario. We will focus on analyzing a continuous stream of product review messages that we collect from all available online channels. Customer-product feedback appears everywhere, including social media channels, partner websites, and customer support systems. We need to capture this valuable customer sentiment about our products as quickly as possible to spot trends and react fast.

With streaming analytics and machine learning, we are able to analyze continuous data streams such as application logs, social media feeds, ecommerce transactions, customer support tickets, and product reviews. For example, we may want to detect quality issues by analyzing real-time product reviews.

In a first step, we will analyze the sentiment of the customer, so we can identify which customers might need high-priority ...