December 2018
Beginner to intermediate
636 pages
18h 15m
English
Content preview from Architecting Modern Data Platforms
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Chapter 6. Datacenter Considerations
Hadoop architects and engineers are not primarily concerned with the datacenter, which offers rather mechanical and commoditized layers of enterprise IT. However, some key features of Hadoop only work as advertised if they are met by the correct layout of datacenter technology. You need to be aware of these effects when placing Hadoop into an existing enterprise IT environment, which has typically been optimized for virtualized host environments and remote storage solutions over the course of the last 10 years.
The content in this chapter, although not an exhaustive discussion on datacenters, may well be of crucial importance for certain key architectural decisions related to reliability and disaster tolerance.
We initially focus on some basic datacenter infrastructure concepts before we revisit some of the ways in which Hadoop differs from other commodity infrastructure setups. We provide a section that addresses common issues with data ingest in the context of datacenters, and finally we highlight common pitfalls that emerge around topics like multidatacenter disaster tolerance.
If you run Hadoop on a public cloud service, much of this chapter is not relevant to your situation, but “Quorum spanning with three datacenters” specifically covers an important subject around cluster spanning that you should observe.
Why Does It Matter ?
Intuition tells us that the distributed nature of Hadoop is likely to have ramifications for the datacenter. ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.