© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2022
S. HainesModern Data Engineering with Apache Sparkhttps://doi.org/10.1007/978-1-4842-7452-1_6

6. Data Discovery and the Spark SQL Catalog

Scott Haines1  
(1)
San Jose, CA, USA
 

Being able to connect and work with the data systems and services that are included in most companies’ modern tech stack is a critical skill for data engineers. Lucky for you, Spark provides the mechanisms to work with and transform data so you can take action and solve problems, instead of writing and maintaining yet another piece of custom infrastructure code. By relying on the core capabilities of Spark, you learned to harness the power of JDBC to interoperate with data stored in a ...

Get Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.