Skip to Main Content
Apache Hive Essentials - Second Edition
book

Apache Hive Essentials - Second Edition

by Dayong Du
June 2018
Beginner to intermediate content levelBeginner to intermediate
210 pages
5h 12m
English
Packt Publishing
Content preview from Apache Hive Essentials - Second Edition

SerDe

SerDe stands for Serialization and Deserialization. It is the technology used to process records and map them to column data types in Hive tables. To explain the scenario of using SerDe, we need to understand how Hive reads and writes data first.

The process to read data is as follows.

  1. Data is read from HDFS.
  2. Data is processed by the INPUTFORMAT implementation, which defines the input data split and key/value records. In Hive, we can use CREATE TABLE ... STORED AS <FILE_FORMAT> (see Chapter 9Performance Considerations) to specify which INPUTFORMAT it reads from.
  3. The Java Deserializer class defined in SerDe is called to format the data into a record that maps to column and data types in a table.

For an example of reading data, we ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Apache Hive Essentials

Apache Hive Essentials

Dayong Du
Apache Hive Cookbook

Apache Hive Cookbook

Hanish Bansal, Saurabh Chauhan, Shrey Mehrotra
Apache Spark Quick Start Guide

Apache Spark Quick Start Guide

Shrey Mehrotra, Akash Grade

Publisher Resources

ISBN: 9781788995092Supplemental Content