book

Apache Hive Cookbook

by Hanish Bansal, Saurabh Chauhan, Shrey Mehrotra

April 2016

Beginner

268 pages

5h 32m

English

Packt Publishing

Read now

Unlock full access

eBooks, discount offers, and moreWhy Subscribe?
What this book covers

Getting readyHow to do it…How it works…There's more…See also
Downloading the example codeDownloading the color images of this bookErrataPiracyQuestions
Introduction
Getting readyHow to do it...How it works…
Getting readyHow to do it…
Getting readyHow to do it…Hive with an embedded metastoreHive with a local metastoreHive with a remote metastore
Getting readyHow to do it...
HiveServerHive metastoreHow to do it...HiveServer2How to do it...Hive clientsHive CLIGetting readyHow to do it...BeelineGetting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
How to do it...
Introducing HiveServer2How to do it…How it works…See also
How to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…BeelineBeeline command optionsJDBCJDBC client sample code using EclipseRunning the JDBC sample code from the command-lineJDBC datatypesOther clients
How to do it…How it works…
How to do it…
Getting readyHow to do it…Prepare dependenciesDownloading and installing HueConfiguring Hive with HueStarting HueAccessing Hive with Hue
IntroductionIntroducing data typesPrimitive data typesComplex data types
How to do it…
How to do it…How it works…
How to do it…
How to do it…
How to do it…
Using relational operatorsHow to do it…Using arithmetic operatorsHow to do it…Using logical operatorsHow to do it…Using complex operatorsHow to do it…
Getting readyHow to do it…
How to do it…Adding new partitionsRenaming partitionsExchanging partitionsDropping the partitionsLoading data in a managed partitioned table
How to do it…
Getting readyHow to do it…How it works…
Introduction
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
How to do it…Create table LIKEHow it works
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…HCatalog DMLs
Getting readyHow to do it…See also…
Introduction
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it...How it works…There's more…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…How it works…There's more…
Getting readyHow to do it...How it works…There's more…
Getting readyHow to do it...How it works…
Introduction
How to do it…LazySimpleSerDeRegexSerDeJSONSerDeCSVSerDeThere's more…See also
How to do it…How it works…
How to do it…
How to do it…Static partitioningDynamic partitioning
How to do it…Metastore view of bucketing
How to do it…See also
How to do it…LEADLAGFIRST_VALUELAST_VALUESee also
How to do it…
Understanding the joins conceptGetting readyHow to do it…How it works…
How to do it…How it works…
How to do it…How it works…
How to do it…How it works…
How to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
How to do it…How it works…
Bringing statistics in to HiveHow to do it…
Getting readyHow to do it…Statistics for a partitioned table
How to do it…How it works…
How to do it…
Using built-in functionsHow to do it…Mathematical functionsCollection functionsType conversion functionsDate functionsString functionsHow it works…Mathematical functionsCollection functionsType conversion functionsDate functionsString functionsThere's moreConditional functionsMiscellaneous functionsSee also
How to do it…How it works…See more
How to do it…How it works…See also
How to do it…How it works…
Enabling predicate pushdown optimizations in HiveGetting readyHow to do it…How it works…
Getting readyHow to do it…
Getting readySampling bucketed tableBlock samplingLength literalRow countHow to do it…How it works…
Securing HadoopHow to do it…How it works…Giving read and write access to user mikeRevoking the access of the user mikeSee also
How to do it…Default authorization–legacy modeStorage-based authorizationSQL standards-based authorizationThere's more
Getting StartedHow to do it…To list out all existing rolescreating a roleDeleting a roleShowing list of current rolesSetting a roleGranting a roleRevoking a roleChecking roles of a user/roleChecking principles of a roleGranting privilegesRevoking privilegesChecking privileges of a user or roleSee also
How to do it…Anonymous with SASL (default no authentication)Anonymous without SASLKerberosConfiguring the JDBC client for Kerberos authenticationAccess Hive using the Beeline clientAccess Hive using the Hive JDBC client in JavaLDAPPluggable Authentication ModulesCustom
Working with Apache SparkGetting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…

Content preview from Apache Hive Cookbook

Optimizations to reduce the number of map

In this recipe, you will learn how to reduce the number of mappers in Hive.

Getting ready

The number of mappers that is used in a map reduce job depends heavily on the input split. The number of mappers is directly proportional to the number of HDFS blocks, that is, the total number of blocks for the input files. Input split is a logical concept that is used to control the number of mappers. If there is no size defined for an input split in map reduce job, then the number of mappers will be equal to the number of HDFS blocks.

However, if you have defined a particular size for an input split, then the number of mappers will be equal to the number of input splits in the MapReduce job and not to the number of ...