Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022

S. HainesModern Data Engineering with Apache Sparkhttps://doi.org/10.1007/978-1-4842-7452-1_15

15. Deploying Mission-Critical Spark Applications on Kubernetes

Scott Haines¹  

(1)

San Jose, CA, USA

 

Before diving headfirst into what Kubernetes (K8s) is, and how Apache Spark fits into the distributed K8s ecosystem, it is important to first begin by stating simply that Kubernetes enables Apache Spark applications to run in isolation, pairing elastic scalability with the runtime consistency of containers, collocated in independent micro-environments called pods (which you’ll learn about soon). Ultimately, you can rely on consistent runtime environments, without ...