book

Apache Hive Cookbook

by Hanish Bansal, Saurabh Chauhan, Shrey Mehrotra

April 2016

Beginner

268 pages

5h 32m

English

Packt Publishing

Read now

Unlock full access

eBooks, discount offers, and moreWhy Subscribe?
What this book covers

Getting readyHow to do it…How it works…There's more…See also
Downloading the example codeDownloading the color images of this bookErrataPiracyQuestions
Introduction
Getting readyHow to do it...How it works…
Getting readyHow to do it…
Getting readyHow to do it…Hive with an embedded metastoreHive with a local metastoreHive with a remote metastore
Getting readyHow to do it...
HiveServerHive metastoreHow to do it...HiveServer2How to do it...Hive clientsHive CLIGetting readyHow to do it...BeelineGetting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
Getting readyHow to do it...
How to do it...
Introducing HiveServer2How to do it…How it works…See also
How to do it…How it works…See also
Getting readyHow to do it…How it works…See also
Getting readyHow to do it…BeelineBeeline command optionsJDBCJDBC client sample code using EclipseRunning the JDBC sample code from the command-lineJDBC datatypesOther clients
How to do it…How it works…
How to do it…
Getting readyHow to do it…Prepare dependenciesDownloading and installing HueConfiguring Hive with HueStarting HueAccessing Hive with Hue
IntroductionIntroducing data typesPrimitive data typesComplex data types
How to do it…
How to do it…How it works…
How to do it…
How to do it…
How to do it…
Using relational operatorsHow to do it…Using arithmetic operatorsHow to do it…Using logical operatorsHow to do it…Using complex operatorsHow to do it…
Getting readyHow to do it…
How to do it…Adding new partitionsRenaming partitionsExchanging partitionsDropping the partitionsLoading data in a managed partitioned table
How to do it…
Getting readyHow to do it…How it works…
Introduction
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
How to do it…Create table LIKEHow it works
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…HCatalog DMLs
Getting readyHow to do it…See also…
Introduction
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it...How it works…There's more…
Getting readyHow to do it…
Getting readyHow to do it…
Getting readyHow to do it…How it works…There's more…
Getting readyHow to do it...How it works…There's more…
Getting readyHow to do it...How it works…
Introduction
How to do it…LazySimpleSerDeRegexSerDeJSONSerDeCSVSerDeThere's more…See also
How to do it…How it works…
How to do it…
How to do it…Static partitioningDynamic partitioning
How to do it…Metastore view of bucketing
How to do it…See also
How to do it…LEADLAGFIRST_VALUELAST_VALUESee also
How to do it…
Understanding the joins conceptGetting readyHow to do it…How it works…
How to do it…How it works…
How to do it…How it works…
How to do it…How it works…
How to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
How to do it…How it works…
Bringing statistics in to HiveHow to do it…
Getting readyHow to do it…Statistics for a partitioned table
How to do it…How it works…
How to do it…
Using built-in functionsHow to do it…Mathematical functionsCollection functionsType conversion functionsDate functionsString functionsHow it works…Mathematical functionsCollection functionsType conversion functionsDate functionsString functionsThere's moreConditional functionsMiscellaneous functionsSee also
How to do it…How it works…See more
How to do it…How it works…See also
How to do it…How it works…
Enabling predicate pushdown optimizations in HiveGetting readyHow to do it…How it works…
Getting readyHow to do it…
Getting readySampling bucketed tableBlock samplingLength literalRow countHow to do it…How it works…
Securing HadoopHow to do it…How it works…Giving read and write access to user mikeRevoking the access of the user mikeSee also
How to do it…Default authorization–legacy modeStorage-based authorizationSQL standards-based authorizationThere's more
Getting StartedHow to do it…To list out all existing rolescreating a roleDeleting a roleShowing list of current rolesSetting a roleGranting a roleRevoking a roleChecking roles of a user/roleChecking principles of a roleGranting privilegesRevoking privilegesChecking privileges of a user or roleSee also
How to do it…Anonymous with SASL (default no authentication)Anonymous without SASLKerberosConfiguring the JDBC client for Kerberos authenticationAccess Hive using the Beeline clientAccess Hive using the Hive JDBC client in JavaLDAPPluggable Authentication ModulesCustom
Working with Apache SparkGetting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…
Getting readyHow to do it…How it works…

Content preview from Apache Hive Cookbook

Top K statistics in Hive

It is the mechanism of collecting the top K column values of a Hive table. In this, the top K values of the most skewed column are stored in the partition. This is applicable for both existing and newly created tables.

How to do it…

Top K statistics computation is disabled by default. The following are some of the properties that could be set to compute and store top K statistics:

hive.stats.topk.collect
This would enable computing top K and putting it into skewed information:
- Default Value: false
- Valid Values: true, false
hive.stats.topk.num
- Using this property, you can specify K value for your top K result
hive.stats.topk.minpercent
- It is the minimal percentage of a row value to be in top K result
- It could be any float value between ...