Similar to partitioning, a bucket table organizes data into separate files in HDFS. Bucketing can speed up data sampling on buckets. Bucketing can also improve join performance if the join keys are also bucket columns because bucketing ensures the keys are present in a certain bucket. Better-chosen bucket columns make a bucket table join perform better. The best practice for choosing bucket columns is to identify the columns that are most likely used in the filter or join condition in terms of the business logic behind the datasets. For more details, refer to the Job optimization section later in this chapter.
Bucket table design
Get Apache Hive Essentials now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
