Apache Hudi: The Definitive Guide

Chapter 1. Read from Hudi

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

As engineers and architects we’ve learned (oftentimes the hard way) that databases are all about tradeoffs. One of the most difficult parts of choosing a database is knowing that no single database can do everything, and no matter which one you pick, it will make implementing certain features much more difficult.

When writing to a traditional relational database (Oracle, PostgreSQL, MySQL, SQLite, etc.) we prepare ourselves for more up-front data engineering work. In return, we expect straightforward queries and transactional guarantees. We know document databases (Mongo, Solr, CouchDB, etc) and key-value stores (e.g. Cassandra, HBase, Redis, RocksDB) will make writes a breeze, and scale horizontally but will eventually lead to headaches when it comes to tighter transactional guarantees and need for higher data consistency between multiple tables.

So we make architectural decisions in support of one business interest at the expense of others. For instance, when the ...