Data Lake for Enterprises

Batch layer

The batch layer is where raw data is stored as is in the rawest format possible. Since no omission nor transformation happens while storing, many different use cases with different perspectives can be derived from these at different stages. This is the store where master data in an immutable state is also available and used by various analyses going forward. Since the data is immutable, update and even delete are forbidden operations. Data is always appended (added) with a timestamp so that when some data is required, it can be queried with the highest timestamp to get the latest record. Delete is also forbidden because then many analyses would require these deleted record details.

The queries, when run against raw data, would ...