book

Learning and Operating Presto

by Angelica Lo Duca, Tim Meehan, Vivek Bharathan, Ying Su

September 2023

Intermediate to advanced

191 pages

4h 32m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Why We Wrote This BookWho This Book Is ForConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgmentsAngelica Lo DucaTim MeehanVivek BharathanYing Su
Data Warehouses and Data LakesThe Role of Presto in a Data LakePresto Origins and Design ConsiderationsHigh PerformanceHigh ScalabilityCompliance with the ANSI SQL StandardFederation of Data SourcesRunning in the CloudPresto Architecture and Core ComponentsAlternatives to PrestoApache ImpalaApache HiveSpark SQLTrinoPresto Use CasesReporting and DashboardingAd Hoc QueryingETL Using SQLData LakehouseReal-Time Analytics with Real-Time DatabasesIntroducing Our Case StudyConclusion
Presto Manual InstallationRunning Presto on DockerInstalling DockerPresto Docker ImageBuilding and Running Presto on DockerThe Presto SandboxDeploying Presto on KubernetesIntroducing KubernetesConfiguring Presto on KubernetesAdding a New CatalogRunning the Deployment on KubernetesQuerying Your Presto InstanceListing CatalogsListing SchemasListing TablesQuerying a TableConclusion
Service Provider InterfaceConnector ArchitecturePopular ConnectorsThriftWriting a Custom ConnectorPrerequisitesPlugin and ModuleConfigurationMetadataInput/OutputDeploying Your ConnectorApache PinotSetting Up and Configuring PrestoPresto-Pinot Querying in ActionConclusion
Setting Up the EnvironmentPresto ClientDocker ImageKubernetes NodeConnectivity to PrestoREST APIPythonRJDBCNode.jsODBCOther Presto Client LibrariesBuilding a Client Dashboard in PythonSetting Up the ClientBuilding the DashboardConclusion
The Emergence of the LakehouseData Lakehouse ArchitectureData LakeFile StoreFile FormatTable FormatQuery EngineMetadata ManagementData GovernanceData Access ControlBuilding a Data LakehouseConfiguring MinIOConfiguring HMSConfiguring SparkRegistering Hudi Tables with HMSConnecting and Querying PrestoConclusion
Introducing Presto AdministrationConfigurationPropertiesSessionsJVMMonitoringConsoleREST APIMetricsManagementResource GroupsVerifiersSession Properties ManagersNamespace FunctionsConclusion
Introducing Presto SecurityBuilding Secure Communication in PrestoEncryptionKeystore ManagementConfiguring HTTPS/TLSAuthenticationFile-Based AuthenticationLDAPKerberosCreating a Custom AuthenticatorAuthorizationAuthorizing Access to the Presto REST APIConfiguring System Access ControlAuthorization Through Apache RangerConclusion
Introducing Performance TuningReasons for Performance TuningThe Performance Tuning Life CycleQuery Execution ModelApproaches for Performance Tuning in PrestoResource AllocationStorageQuery OptimizationAria ScanTable ScanningRepartitioningImplementing Performance TuningBuilding and Importing the Sample CSV Table in MinIOConverting the CSV Table in ORCDefining the Tuning ParametersRunning TestsConclusion
Introducing ScalabilityReasons to Scale PrestoCommon IssuesDesign ConsiderationsAvailabilityManageabilityPerformanceProtectionConfigurationHow to Scale PrestoMultiple CoordinatorsPresto on SparkSpillingUsing a Cloud ServiceConclusion

Content preview from Learning and Operating Presto

Chapter 2. Getting Started with Presto

There are various ways to set up Presto. In the first part of this chapter, you’ll see how to install Presto manually. We won’t dwell much on manual installation, because the real goal of the chapter is to build a small cluster locally, which simulates a realistic production environment.

Next, you’ll see how to install Presto on Docker and how to set up a local cluster in Kubernetes. Finally, you’ll learn how to run your first queries in Presto, with a focus on how to list catalogs and schemas, and query a table.

This chapter assumes that you have some familiarity with Docker, Kubernetes, and using a command line. Even if you don’t, you should still be able to follow the instructions end to end and get a working deployment running successfully.

Presto Manual Installation

To set up Presto on your local machine, you can follow the procedure described in the Presto official documentation. Since Presto is written in Java, you should pay attention to the specific versions of Java supported by Presto. Presto requires Java 8. Both Oracle JDK and OpenJDK are supported.

Running Presto on Docker

The local Presto instance is great for development and troubleshooting. You can set up a full production cluster with multiple nodes using bare-metal machines or VMs. However, you would need to manage the monitoring, availability, scalability, and rollout of these nodes by yourself. This is where containers and Kubernetes come in.

In this section, we’ll ...