3
Data Management with Delta Lake
Delta Lake is an open source storage layer that enables building a lakehouse architecture with various compute engines and APIs. It provides features such as atomicity, consistency, isolation, and durability (ACID) transactions, scalable metadata, time travel, schema evolution, and data manipulation language (DML) operations. It is compatible with Apache Spark and other query engines.
This chapter provides a comprehensive overview of how to manage and optimize Delta tables using Apache Spark. It covers topics such as creating Delta tables, querying and analyzing them, optimizing them for better performance and cost-effectiveness, managing table metadata, migrating data to Delta Lake, and versioning Delta tables ...
Get Data Engineering with Databricks Cookbook now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
