Simplifying Data Engineering and Analytics with Delta

Chapter 5: Data Consolidation in Delta Lake

"Faith makes you stable and steady. It brings out the totality in you.

Consolidation of your energy is faith. Dissemination of energy is doubt."

– Sri Sri Ravi Shankar, on Transcendental Meditation

In the previous chapters, we discussed the quality of Delta and why it has become the first choice in big data processing. In this chapter, we will focus on how to consolidate disparate datasets into one or more data lakes backed by Delta so that you can build all kinds of use cases on a single source of truth without having to move data or stitch together multiple systems. We have already looked into the special features that Delta offers, including ACID transaction support, schema evolution, time travel, ...