May 2017
Beginner to intermediate
596 pages
15h 2m
English
Content preview from Data Lake for Enterprises
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Batch process layer
A batch process, from the perspective of data access, requires sequential access of data and the storage layer should be optimized for this operation. Hadoop as a technology reads and writes data as blocks and each of these blocks contains data in a sequence. Even the block level access in Hadoop is in a sequence to make sure that any batch to disk operation is fast enough even on spinning disks (commodity hardware).