May 2024
Intermediate to advanced
110 pages
1h 19m
Chinese
Content preview from 面向MapReduce的Hadoop优化Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
译者序
毫无疑问，我们正处在非凡的大数据时代而且参与其中。我们每一个人，要么是数据产生的源头，要么是数据生产者、消费者或者加工者。长期以来，各种数据，包括非结构化数据、半结构化数据和传统的结构化数据一直就在那里，数据科学家为我们打开了大数据的价值之门，而存储和发掘大数据价值金矿的任务是数据工程师的使命和机遇。
MapReduce作为一个成功的大型分布式并行编程模型得到了大数据工程师的广泛认同，而Hadoop作为MapReduce的一个实现，受到广泛的欢迎并被应用到很多的数据开发领域，而且在大数据时代成为数据工程师的首选。
Hadoop包括了分布式存储引擎和分布式计算引擎，即我们熟知的HDFS和Hadoop MapReduce。HDFS使得任意规模数据的存储变得简单、可靠且代价低廉，而MapReduce编程框架更为数据分析提供了极大便利，开发人员仅需把用户逻辑写进 map()和reduce()两个函数，就可以实现分布式并行运算。
尽管 Hadoop 为数据工程师入门和编程提供了极大便利，但构造一个真正满足需求的MapReduce程序并不简单。数据量巨大是大数据工作的现实问题，而性能尤其是运行时间经常是主要困扰。虽然map()和reduce()函数承载了用户执行逻辑，但这远不是MapReduce程序的全部。更多的程序逻辑包含在MapReduce编程框架和HDFS中，而MapReduce框架和HDFS更依赖集群的系统配置。要使MapReduce程序高效运行，除了考虑mapper和reducer代码优化，更要进行MapReduce框架优化，根据需要进行数据、逻辑和框架的适配。
本书是一本不可多得的 MapReduce 性能调节和优化手册，向读者说明影响MapReduce性能的因素，展示监测性能和改善性能的技术手段，并针对特定的集群实例，一步一步教读者如何获取基线，如何计算参数的恰当取值，如何进行优化迭代，直到获得理想的优化效果。 ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.