Trino: The Definitive Guide, 2nd Edition

Trino: The Definitive Guide, 2nd Edition

by Matt Fuller, Manfred Moser, Martin Traverso
Released October 2022
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098137236

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Perform fast interactive analytics against different data sources using the Trino high-performance distributed SQL query engine. In the second edition of this practical guide, you'll learn how to conduct analytics on data where it lives, whether it's a data lake using Hive, a modern lakehouse with Iceberg or Delta Lake, a different system like Cassandra, Kafka, or SingleStore, or a relational database like PostgreSQL or Oracle.

Analysts, software engineers, and production engineers learn how to manage, use, and even develop with Trino and make it a critical part of their data platform. Authors Matt Fuller, Manfred Moser, and Martin Traverso show you how a single Trino query can combine data from multiple sources to allow for analytics across your entire organization.

  • Explore Trino's use cases, and learn about tools that help you connect to Trino for querying and processing huge amounts of data
  • Learn Trino's internal workings, including how to connect to and query data sources with support for SQL statements, operators, functions, and more
  • Deploy and secure Trino at scale, monitor workloads, tune queries, and connect more applications
  • Learn how other organizations apply Trino successfully

Publisher resources

View/Submit Errata

Table of contents

  1. Foreword
  2. Preface
    1. Conventions Used in This Book
    2. Code Examples, Permissions, and Attribution
    3. O’Reilly Online Learning
    4. How to Contact Us
    5. Acknowledgments
  3. I. Getting Started with Trino
  4. 1. Introducing Trino
    1. The Problems with Big Data
    2. Trino to the Rescue
      1. Designed for Performance and Scale
      2. SQL-on-Anything
      3. Separation of Data Storage and Query Compute Resources
    3. Trino Use Cases
      1. One SQL Analytics Access Point
      2. Access Point to Data Warehouse and Source Systems
      3. Provide SQL-Based Access to Anything
      4. Federated Queries
      5. Semantic Layer for a Virtual Data Warehouse
      6. Data Lake Query Engine
      7. SQL Conversions and ETL
      8. Better Insights Due to Faster Response Times
      9. Big Data, Machine Learning, and Artificial Intelligence
      10. Other Use Cases
    4. Trino Resources
      1. Website
      2. Documentation
      3. Community Chat
      4. Source Code, License, and Version
      5. Contributing
      6. Book Repository
      7. Iris Data Set
      8. Flight Data Set
    5. A Brief History of Trino
    6. Conclusion
  5. 2. Installing and Configuring Trino
    1. Trying Trino with the Docker Container
    2. Installing from the Archive File
      1. Java Virtual Machine
      2. Python
      3. Installation
      4. Configuration
    3. Adding a Data Source
    4. Running Trino
    5. Conclusion
  6. 3. Using Trino
    1. Trino Command-Line Interface
      1. Getting Started
      2. Pagination
      3. History and Completion
      4. Additional Diagnostics
      5. Executing Queries
      6. Output Formats
      7. Ignoring Errors
    2. Trino JDBC Driver
      1. Downloading and Registering the Driver
      2. Establishing a Connection to Trino
    3. Trino and ODBC
    4. Client Libraries
    5. Trino Web UI
    6. SQL with Trino
      1. Concepts
      2. First Examples
    7. Conclusion
  7. II. Diving Deeper into Trino
  8. 4. Trino Architecture
    1. Coordinator and Workers in a Cluster
      1. Coordinator
      2. Discovery Service
      3. Workers
    2. Connector-Based Architecture
    3. Catalogs, Schemas, and Tables
    4. Query Execution Model
    5. Query Planning
      1. Parsing and Analysis
      2. Initial Query Planning
    6. Optimization Rules
      1. Predicate Pushdown
      2. Cross Join Elimination
      3. TopN
      4. Partial Aggregations
    7. Implementation Rules
      1. Lateral Join Decorrelation
      2. Semi-Join (IN) Decorrelation
    8. Cost-Based Optimizer
      1. The Cost Concept
      2. Cost of the Join
      3. Table Statistics
      4. Filter Statistics
      5. Table Statistics for Partitioned Tables
      6. Join Enumeration
      7. Broadcast Versus Distributed Joins
    9. Working with Table Statistics
      1. Trino ANALYZE
      2. Gathering Statistics When Writing to Disk
      3. Hive ANALYZE
      4. Displaying Table Statistics
    10. Conclusion
  9. 5. Production-Ready Deployment
    1. Configuration Details
    2. Server Configuration
    3. Logging
    4. Node Configuration
    5. JVM Configuration
    6. Launcher
    7. Cluster Installation
    8. RPM Installation
      1. Installation Directory Structure
      2. Configuration
      3. Uninstall Trino
    9. Installation in the Cloud
    10. Helm Chart for Kubernetes Deployment
    11. Cluster Sizing Considerations
    12. Conclusion
  10. 6. Connectors
    1. Configuration
    2. RDBMS Connector Example: PostgreSQL
      1. Query Pushdown
      2. Parallelism and Concurrency
      3. Other RDBMS Connectors
      4. Security
      5. Query Pass-Through
    3. Trino TPC-H and TPC-DS Connectors
    4. Hive Connector for Distributed Storage Data Sources
      1. Apache Hadoop and Hive
      2. Hive Connector
      3. Hive-Style Table Format
      4. Managed and External Tables
      5. Partitioned Data
      6. Loading Data
      7. File Formats and Compression
      8. MinIO Example
    5. Modern Distributed Storage Management and Analytics
    6. Non-Relational Data Sources
    7. Trino JMX Connector
    8. Black Hole Connector
    9. Memory Connector
    10. Other Connectors
    11. Conclusion
  11. 7. Advanced Connector Examples
    1. Connecting to HBase with Phoenix
    2. Key-Value Store Connector Example: Accumulo
      1. Using the Trino Accumulo Connector
      2. Predicate Pushdown in Accumulo
    3. Apache Cassandra Connector
    4. Streaming System Connector Example: Kafka
    5. Document Store Connector Example: Elasticsearch
      1. Overview
      2. Configuration and Usage
      3. Query Processing
      4. Full-Text Search
      5. Summary
    6. Query Federation in Trino
    7. Extract, Transform, Load and Federated Queries
    8. Conclusion
  12. 8. Using SQL in Trino
    1. Trino Statements
    2. Trino System Tables
    3. Catalogs
    4. Schemas
    5. Information Schema
    6. Tables
      1. Table and Column Properties
      2. Copying an Existing Table
      3. Creating a New Table from Query Results
      4. Modifying a Table
      5. Deleting a Table
      6. Table Limitations from Connectors
    7. Views
    8. Session Information and Configuration
    9. Data Types
      1. Collection Data Types
      2. Temporal Data Types
      3. Type Casting
    10. SELECT Statement Basics
    11. WHERE Clause
    12. GROUP BY and HAVING Clauses
    13. ORDER BY and LIMIT Clauses
    14. JOIN Statements
    15. UNION, INTERSECT, and EXCEPT Clauses
    16. Grouping Operations
    17. WITH Clause
    18. Subqueries
      1. Scalar Subquery
      2. EXISTS Subquery
      3. Quantified Subquery
    19. Deleting Data from a Table
    20. Conclusion
  13. 9. Advanced SQL
    1. Functions and Operators Introduction
    2. Scalar Functions and Operators
    3. Boolean Operators
    4. Logical Operators
    5. Range Selection with the BETWEEN Statement
    6. Value Detection with IS (NOT) NULL
    7. Mathematical Functions and Operators
    8. Trigonometric Functions
    9. Constant and Random Functions
    10. String Functions and Operators
    11. Strings and Maps
    12. Unicode
    13. Regular Expressions
    14. Unnesting Complex Data Types
    15. JSON Functions
    16. Date and Time Functions and Operators
    17. Histograms
    18. Aggregate Functions
      1. Map Aggregate Functions
      2. Approximate Aggregate Functions
    19. Window Functions
    20. Lambda Expressions
    21. Geospatial Functions
    22. Prepared Statements
    23. Conclusion
  14. III. Trino in Real-World Uses
  15. 10. Security
    1. Authentication
      1. Password and LDAP Authentication
      2. Other Authentication Types
    2. Authorization
      1. System Access Control
      2. Connector Access Control
    3. Encryption
      1. Encrypting Trino Client-to-Coordinator Communication
      2. Creating Java Keystores and Java Truststores
      3. Encrypting Communication Within the Trino Cluster
    4. Certificate Authority Versus Self-Signed Certificates
    5. Certificate Authentication
    6. Kerberos
      1. Prerequisites
      2. Kerberos Client Authentication
    7. Data Source Access and Configuration for Security
    8. Kerberos Authentication with the Hive Connector
      1. Hive Metastore Service Authentication
      2. HDFS Authentication
    9. Cluster Separation
    10. Conclusion
  16. 11. Integrating Trino with Other Tools
    1. Queries, Visualizations, and More with Apache Superset
    2. Performance Improvements with RubiX
    3. Workflows with Apache Airflow
    4. Embedded Trino Example: Amazon Athena
    5. Convenient Commercial Distributions: Starburst Enterprise and Starburst Galaxy
    6. Other Integration Examples
    7. Custom Integrations
    8. Conclusion
  17. 12. Trino in Production
    1. Monitoring with the Trino Web UI
      1. Cluster-Level Details
      2. Query List
      3. Query Details View
    2. Tuning Trino SQL Queries
    3. Memory Management
    4. Task Concurrency
    5. Worker Scheduling
    6. Network Data Exchange
      1. Concurrency
      2. Buffer Sizes
    7. Tuning Java Virtual Machine
    8. Resource Groups
      1. Resource Group Definition
      2. Scheduling Policy
      3. Selector Rules Definition
    9. Conclusion
  18. 13. Real-World Examples
    1. Deployment and Runtime Platforms
    2. Cluster Sizing
    3. Hadoop/Hive Migration Use Case
    4. Other Data Sources
    5. Users and Traffic
    6. Conclusion
  19. Conclusion
  20. Index
  21. About the Authors

Product information

  • Title: Trino: The Definitive Guide, 2nd Edition
  • Author(s): Matt Fuller, Manfred Moser, Martin Traverso
  • Release date: October 2022
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098137236