May 2024
Intermediate to advanced
110 pages
1h 19m
Chinese
Content preview from 面向MapReduce的Hadoop优化Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第7章 最佳实践与建议
好吧，这一章是一个精彩的结尾！截至目前，我们学习了MapReduce作业性能的优化方法，并利用很大篇幅对重要的知识点作了铺垫。请记住，建立 Hadoop 集群的基本挑战在于要综合考量高可用性、负载均衡以及因此而获得个性化需求服务。
在本章中，我们介绍用来优化Hadoop MapReduce作业的硬件及应用配置检查表。
本章将介绍以下内容：
常见的Hadoop 集群检查表；
BIOS 检查表及OS 推荐；
Hadoop 最佳实践与建议；
应用程序中用到的MapReduce 模板类。
7.1 硬件调优与操作系统推荐
系统调优推荐措施与系统的内在能力紧密相关。接下来的几节将给出不同的技术建议与提示，在进行MapReduce优化过程中，这些建议与提示可作为提醒的基线。
7.1.1 Hadoop集群检查表
下面的检查表是使Hadoop集群工作最优所需的最低要求。
检查并确保所有的集群节点都能互相通信，并且能够有物理手段和/或远程管理手段访问每个集群节点。
检查集群是否正确规划以及是否能（至少）对每个服务的一个节点的故障进行补偿。
检查集群环境的限制（硬件可用性资源/机架空间、主机参数等）。
定义集群的失效接管策略，确保服务的高可用性。
定义需要备份的内容，以及需要保存什么和在哪里保存，以便让 Hadoop 存储容量最大化。
7.1.2 Bios调优检查表
下面列出了安装在优化环境下使用的Hadoop集群节点时需要检查的项目。
检查硬件上的所有 CPU 内核是否得到了充分利用；否则，你可以降级 CPU频率。
启用原生命令排队模式（NCQ），有助于通过优化驱动器磁头移动来改善主流硬盘驱动器的 I/O 性能。一般来说，可以通过 BIOS 中的 Advanced Host ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.