book

Apache Hive Cookbook

by Hanish Bansal, Saurabh Chauhan, Shrey Mehrotra

April 2016

Beginner

268 pages

5h 32m

English

Packt Publishing

Read now

Unlock full access

eBooks, discount offers, and moreWhy Subscribe?
What this book covers

Getting readyHow to do it…How it works…There's more…See also
Downloading the example codeDownloading the color images of this bookErrataPiracyQuestions
Introduction
Getting readyHow to do it...How it works…
Getting readyHow to do it…
Getting readyHow to do it…Hive with an embedded metastoreHive with a local metastoreHive with a remote metastore
Getting readyHow to do it...
HiveServerHive metastoreHow to do it...HiveServer2How to do it...Hive clientsHive CLIGetting readyHow to do it...BeelineGetting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
How to do it...
Introducing HiveServer2How to do it…How it works…See also
How to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…BeelineBeeline command optionsJDBCJDBC client sample code using EclipseRunning the JDBC sample code from the command-lineJDBC datatypesOther clients
How to do it…How it works…
How to do it…
Getting readyHow to do it…Prepare dependenciesDownloading and installing HueConfiguring Hive with HueStarting HueAccessing Hive with Hue
IntroductionIntroducing data typesPrimitive data typesComplex data types
How to do it…
How to do it…How it works…
How to do it…
How to do it…
How to do it…
Using relational operatorsHow to do it…Using arithmetic operatorsHow to do it…Using logical operatorsHow to do it…Using complex operatorsHow to do it…
Getting readyHow to do it…
How to do it…Adding new partitionsRenaming partitionsExchanging partitionsDropping the partitionsLoading data in a managed partitioned table
How to do it…
Getting readyHow to do it…How it works…
Introduction
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
How to do it…Create table LIKEHow it works
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…HCatalog DMLs
Getting readyHow to do it…See also…
Introduction
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it...How it works…There's more…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…How it works…There's more…
Getting readyHow to do it...How it works…There's more…
Getting readyHow to do it...How it works…
Introduction
How to do it…LazySimpleSerDeRegexSerDeJSONSerDeCSVSerDeThere's more…See also
How to do it…How it works…
How to do it…
How to do it…Static partitioningDynamic partitioning
How to do it…Metastore view of bucketing
How to do it…See also
How to do it…LEADLAGFIRST_VALUELAST_VALUESee also
How to do it…
Understanding the joins conceptGetting readyHow to do it…How it works…
How to do it…How it works…
How to do it…How it works…
How to do it…How it works…
How to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
How to do it…How it works…
Bringing statistics in to HiveHow to do it…
Getting readyHow to do it…Statistics for a partitioned table
How to do it…How it works…
How to do it…
Using built-in functionsHow to do it…Mathematical functionsCollection functionsType conversion functionsDate functionsString functionsHow it works…Mathematical functionsCollection functionsType conversion functionsDate functionsString functionsThere's moreConditional functionsMiscellaneous functionsSee also
How to do it…How it works…See more
How to do it…How it works…See also
How to do it…How it works…
Enabling predicate pushdown optimizations in HiveGetting readyHow to do it…How it works…
Getting readyHow to do it…
Getting readySampling bucketed tableBlock samplingLength literalRow countHow to do it…How it works…
Securing HadoopHow to do it…How it works…Giving read and write access to user mikeRevoking the access of the user mikeSee also
How to do it…Default authorization–legacy modeStorage-based authorizationSQL standards-based authorizationThere's more
Getting StartedHow to do it…To list out all existing rolescreating a roleDeleting a roleShowing list of current rolesSetting a roleGranting a roleRevoking a roleChecking roles of a user/roleChecking principles of a roleGranting privilegesRevoking privilegesChecking privileges of a user or roleSee also
How to do it…Anonymous with SASL (default no authentication)Anonymous without SASLKerberosConfiguring the JDBC client for Kerberos authenticationAccess Hive using the Beeline clientAccess Hive using the Hive JDBC client in JavaLDAPPluggable Authentication ModulesCustom
Working with Apache SparkGetting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…

Content preview from Apache Hive Cookbook

Using the built-in User-defined Aggregation Function (UDAF)

Hive provides a set of functions to do aggregation on a dataset. These functions operate on a range of data (rows) and provide the cumulative or relative result.

How to do it…

The built-in functions could be used directly in the query. The following are some of the examples of aggregated functions available in Hive:

Function Name	Return Type	Description
`avg(col)`	DOUBLE	It is used to calculate the average of all values of a particular column.
`avg(DISTINCT col)`	DOUBLE	It is used to calculate the average of unique values of a particular column.
`collect_list(col)`	ARRAY	It will return a list of all values of a particular column in an array.
`collect_set(col)`	ARRAY	It will return ...