Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022

S. HainesModern Data Engineering with Apache Sparkhttps://doi.org/10.1007/978-1-4842-7452-1_12

12. Analytical Processing and Insights

Scott Haines¹  

(1)

San Jose, CA, USA

 

Throughout the book you’ve focused on building your data engineering toolbox and learning powerful techniques for processing and transforming data, both at rest and in stream. Parallel to learning Spark, you have also been hands-on learning to use containers and operate additional technologies that interoperate well in the Apache Spark ecosystem.

Given that the modern data engineer wears many hats and has many responsibilities spanning the wide spectrum of the data ecosystem, from data production ...