Mastering Azure Analytics, 1st Edition

Chapter 1. Enterprise Analytics Fundamentals

In this chapter we’ll review the fundamentals of enterprise analytic architectures. We will introduce the analytics data pipeline, a fundamental process that takes data from its source through several steps until it is available to analytics clients. Then we will introduce the concept of a data lake, as well as two different pipeline architectures: lambda architecture and kappa architecture. The particular steps in the typical data processing pipeline (as well as considerations around the handling of “hot” and “cold” data) are detailed and serve as a framework for the rest of the book. We conclude the chapter by introducing our case study scenarios, along with their respective data sets, which provide a more real-world context for performing big data analytics on Azure.

The Analytics Data Pipeline

Data does not end up nicely formatted for analytics on its own; it takes a series of steps that involve collecting the data from the source, massaging the data to get it into the forms appropriate to the analytics desired (sometimes referred to as data wrangling or data munging), and ultimately pushing the prepared results to the location from which they can be consumed. This series of steps can be thought of as a pipeline.

The analytics data pipeline forms a basis for understanding any analytics solution, and thus is very useful to our purposes in this book as we seek to understand how to accomplish analytics using Microsoft Azure. As shown ...