Delta Lake: Up and Running

Preface

The goal of this book is to provide data practitioners with practical instructions on how to set up Delta Lake and start using its unique features. This book is designed for an audience that fits any of the following profiles:

• Data practitioners with a Spark background

• Data practitioners unfamiliar with or new to Delta Lake needing an introduction to the technology, the problems it solves, its main features and terminology, as well as how to get started using it

• Data practitioners looking to learn about the features and benefits of modern lakehouse architectures

It is important to note that this book and the features discussed apply to the Delta Lake open source framework (Delta Lake OSS). Proprietary features and optimizations that some companies offer around Delta Lake are considered out of the scope of this book.

First, we discuss why Delta Lake is an important tool for building modern enterprise data platforms and data science and AI solutions, followed by instructions on how to set up Delta Lake with Spark. Each of the subsequent chapters will walk you through the fundamental functions and operations of Delta Lake using step-by-step instructions and real-world examples.

The code examples in the book range from snippets that can be used in a PySpark shell to those designed to be run with a complete end-to-end notebook. In this book, all code snippets will be in Python, SQL, and, where necessary, shell commands.

A GitHub repository is provided to aid readers ...