July 2017
Beginner to intermediate
715 pages
17h 3m
English
Content preview from Java: Data Science Made Easy
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Apache Hadoop
Apache Hadoop is a set of tools that allows you to scale your data processing pipelines to thousands of machines. It includes:
- Hadoop MapReduce: This is a data processing framework
- HDFS: This is a distributed filesystem, which allows us to store data on multiple machines
- YARN: This is the executor of MapReduce and other jobs
We will only cover MapReduce, as it is the core of Hadoop and it is related to data processing. We will not cover the rest, and we will also not talk about setting up or configuring a Hadoop Cluster as this is slightly beyond scope for this book. If you are interested in knowing more about it, Hadoop: The Definitive Guide by Tom White is an excellent book for learning this subject in depth.
In our experiments, ...