System Design on AWS

Chapter 11. Big Data, Analytics and Machine Learning Services

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 13th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

In the world of information technology, data is generated at a huge volume. This data can be just the user information of all registered users on online food ordering applications or capturing real-time user actions on the application. The data generated at large volume is referred to as Big Data. If you have a use case to store this data, you can utilize the storage solutions we discussed in Chapter 10 based on your requirements. This chapter focuses on how we can process the data at high volume. How can we generate insights out of data already present in storage or live streaming data by running data analytics or machine learning models on top of it? For example, determining the most ordered food item based on location or restaurants with highest rating in particular locality.

The first part of this chapter introduces you to AWS big data, live streaming and analytics ...