The Definitive Guide to Azure Data Engineering: Modern ELT, DevOps, and Analytics on the Azure Cloud Platform

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2021

R. C. L'EsteveThe Definitive Guide to Azure Data Engineeringhttps://doi.org/10.1007/978-1-4842-7182-7_15

15. Delta Lake

Ron C. L’Esteve¹  

(1)

Chicago, IL, USA

 

While working with Azure Data Lake Storage Gen2 and Apache Spark, users have learned about both the limitations of Apache Spark and the many data lake implementation challenges. The need for an ACID-compliant feature set is critical within the data lake landscape, and Delta Lake offers many solutions to the current limitations of the standard Azure Data Lake Storage Gen2 accounts.

Delta Lake is an open source storage layer that guarantees data atomicity, consistency, isolation, and durability in the lake. ...