DuckLake: The Definitive Guide

Chapter 1. Rethinking the Lakehouse

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at [e-mail].

The Pain Points of Today’s Lakehouses

Imagine setting up a lakehouse attached to a cloud object store in less than a minute. No Apache Spark, no Java, no catalog wiring, no distributed architectural management. Just a couple lines of SQL and you are ready. It might sound absurdly optimistic, but it is not. Ducklake makes this a reality and here’s the proof. With just two lines of SQL, you will have a Ducklake lakehouse wired up to GCS ready to make and manage tables:

CREATE OR REPLACE SECRET gcs_creds (TYPE GCS, KEY_ID getenv('GCS_KEY'), SECRET getenv('GCS_SECRET'));
ATTACH OR REPLACE 'ducklake:gcs_wh.ducklake' AS gcs_wh (DATA_PATH getenv('GCS_WHS_PATH'));

If you have built lakehouses with other formats, then the simplicity illustrated here might be hard to believe. Why does DuckLake take a few lines to code and setup, while Apache Iceberg and Delta Lake require pages of configuration? The answer boils down to a core architectural decision - where the metadata ...