September 2012
Intermediate to advanced
350 pages
9h 46m
English
Content preview from Programming HiveBecome an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Chapter 17. Storage Handlers and NoSQL
Storage Handlers are a combination of InputFormat, OutputFormat, SerDe, and specific code that Hive uses to treat
an external entity as a standard Hive table. This allows the user to issue
queries seamlessly whether the table represents a text file stored in Hadoop
or a column family stored in a NoSQL database such as Apache
HBase, Apache Cassandra, and
Amazon DynamoDB. Storage handlers are not only limited
to NoSQL databases, a storage handler could be designed for many different
kinds of data stores.
Note
A specific storage handler may only implement some of the capabilities. For example, a given storage handler may allow read-only access or impose some other restriction.
Storage handlers offer a streamlined system for ETL. For example, a Hive query could be run that selects a data table that is backed by sequence files, however it could output to text files.
Storage Handler Background
Hadoop has an abstraction known as InputFormat that allows data from different
sources and formats to be used as input for a job. The TextInputFormat is a concrete implementation of
InputFormat. It works by providing
Hadoop with information on how to split a given path into multiple tasks,
and it provides a RecordReader that
provides methods for reading data from each split.
Hadoop also has an abstraction known as OutputFormat, which takes the output from a job
and outputs it to an entity. The TextOutputFormat is a concrete implementation of
OutputFormat. It works ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.