July 2017
Beginner to intermediate
378 pages
10h 26m
English
Content preview from Analytics for the Internet of Things (IoT)
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Hadoop Distributed File System
Hadoop Distributed File System (HDFS) is a filesystem spread across multiple servers and is designed to run on low-cost commodity hardware. HDFS supports a write-once and read many philosophy. It was designed for large-scale batch processing work on large to enormous sized files.
Files are divided up into blocks. A typical block size is 128 MB. A file on HDFS is sliced up into 128 MB chunks (the blocks) and distributed across different data nodes. Files in HDFS normally range from gigabytes to terabytes.
HDFS was designed for batch processing more than low-latency interactive queries from users. HDFS is not meant for files that frequently change with data updates. New data is typically appended to files or added ...