book

Elastic Stack 8.x Cookbook

Name: Elastic Stack 8.x Cookbook
ISBN: 9781837634293

by Huage Chen, Yazid Akadiri

June 2024

Intermediate to advanced

688 pages

15h 24m

English

Packt Publishing

Read now

Unlock full access

Elastic Stack 8.x Cookbook
Foreword
Contributors
About the authorsAbout the reviewers
Acknowledgments
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedSectionsGetting readyHow to do it…How it works…There’s more…See alsoGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Chapter 1: Getting Started – Installing the Elastic Stack
Deploying the Elastic Stack on Elastic CloudHow to do it…How it works…There’s more…Installing the Elastic Stack with ECKTechnical requirementsGetting readyHow to do it…How it works…There’s more…See alsoInstalling a self-managed Elastic StackGetting readyHow to do it…How it works…There’s more…Creating and setting up data tieringGetting readyHow to do it on your local machine…How it works (on self-managed)…How to do it on Elastic Cloud…How to do it on ECK…There’s more…See alsoCreating and setting up additional Elasticsearch nodesGetting readyHow to do it...How it works…How to do it on Elastic Cloud...How to do it on ECK…There’s more…See alsoCreating and setting up Fleet ServerGetting readyHow to do it on a self-managed Elastic Stack…How it works…Setting up on Elastic CloudSee alsoSetting up snapshot repositoryGetting readyHow to do it…How it works…There’s more…
Chapter 2: Ingesting General Content Data
Introducing the Wikipedia Movie Plots datasetTechnical requirementsAdding data from the Elasticsearch clientGetting readyHow to do it…How it works...There’s more…Updating data in ElasticsearchGetting readyHow to do it…How it works...There’s more…Deleting data in ElasticsearchGetting readyHow to do it…How it works...There’s more…See alsoUsing an analyzerGetting readyHow to do it…How it works...There’s more…Defining index mappingGetting readyHow to do it…How it works...There’s more…See alsoUsing dynamic templates in document mappingGetting readyHow to do it…How it works...There’s more…See alsoCreating an index templateGetting readyHow to do it…How it works...There’s more…Indexing multiple documents using Bulk APIGetting readyHow to do it…How it works...There’s more…See also
Chapter 3: Building Search Applications
Technical requirementsSearching with Query DSLGetting readyHow to do it...How it works...There’s more...Building advanced search queries with Query DSLGetting readyHow to do it...How it works...There’s more...See alsoUsing search templates to pre-render search requestsGetting readyHow to do it...How it works...There’s more...See alsoGetting started with Search Applications for your Elasticsearch indexGetting readyHow to do it...How it works...Building a search experience with the Search Application clientGetting readyHow to do it...How it works...There’s more...See alsoMeasuring the performance of your Search Applications with Behavioral AnalyticsGetting readyHow to do it...How it works...There’s more...See also
Chapter 4: Timestamped Data Ingestion
Technical requirementsDeploying Elastic Agent with FleetGetting readyHow to do it...How it works...There’s more...See alsoMonitoring Apache HTTP logs and metrics using the Apache integrationGetting readyHow to do it...How it works...There’s more...See alsoDeploying standalone Elastic AgentGetting readyHow to do it...How it works...There’s more...See alsoAdding data using BeatsGetting readyHow to do it...How it works...There’s more...See alsoSetting up a data stream manuallyDatasetGetting readyHow to do it...How it works...There’s more...See alsoSetting up a time series data stream manuallyGetting readyHow to do it...How it works…There’s more...See also
Chapter 5: Transform Data
Technical requirementsCreating an ingest pipelineGetting readyHow to do it...How it works...There’s more...See alsoEnriching data with a custom ingest pipeline for an existing Elastic Agent integrationGetting readyHow to do it...How it works...There’s more...Using a processor to enrich your data before ingesting with Elastic AgentGetting readyHow to do it...How it works...There’s more...See alsoInstalling self-managed LogstashGetting readyHow to do it...How it works...There’s more...See alsoCreating a Logstash pipelineGetting readyHow to do it...How it works...There’s more...See alsoSetting up pivot data transformGetting readyHow to do it...How it works...There’s more...See alsoSetting up the latest data transformGetting readyHow to do it...How it works...There’s more...See alsoDownsampling your time series dataGetting readyHow to do it...How it works...There’s more...See also

Chapter 6: Visualize and Explore Data
Technical requirementsExploring your data in DiscoverGetting readyHow to do it...How it works...There’s more...See alsoExploring your data with ES|QLGetting readyHow to do it...How it works...There’s more...See alsoCreating visualizations with Kibana LensGetting readyHow to do it...How it works...There’s more...See alsoCreating visualizations from runtime fieldsGetting readyHow to do it...How it works...There’s more...See alsoCreating Kibana mapsGetting readyHow to do it...How it works...There’s more…See alsoCreating and using Kibana dashboardsGetting readyHow to do it...How it works...There’s more...See alsoCreating Canvas workpadsGetting readyHow to do it...How it works...There’s more…See also
Chapter 7: Alerting and Anomaly Detection
Technical requirementsCreating alerts in KibanaGetting readyHow to do it...How it works...There’s more…See alsoMonitoring alert rulesGetting readyHow to do it...How it works...There’s more...See alsoInvestigating data with log rate analysisGetting readyHow to do it...How it works...There’s more…See alsoInvestigating data with log pattern analysisGetting readyHow to do it...How it works...There’s more...Investigating data with change point detectionGetting readyHow to do it...How it works...There’s more...See alsoDetecting anomalies in your data with unsupervised machine learning jobsGetting readyHow to do it...How it works...There’s more...See alsoCreating anomaly detection jobs from a Lens visualizationGetting readyHow to do it...How it works...There’s more...
Chapter 8: Advanced Data Analysis and Processing
Technical requirementsFinding deviations in your data with outlier detectionGetting readyHow to do it…How it works…See alsoBuilding a model to perform regression analysisGetting readyHow to do it…How it works…There’s more…See alsoBuilding a model for classificationGetting readyHow to do it…How it works…There’s more…See alsoUsing a trained model for inferenceGetting readyHow to do it…How it works…There’s more…See alsoDeploying third-party NLP models and testing via the UIGetting readyHow to do it…How it works…There’s more…See alsoRunning advanced data processing with trained modelsGetting readyHow to do it…How it works…There’s more…See also
Chapter 9: Vector Search and Generative AI Integration
Technical requirementsImplementing semantic search with dense vectorsGetting readyHow to do it…How it works…There’s more…See alsoImplementing semantic search with sparse vectorsGetting readyHow to do it...How it works...There’s more...See alsoUsing hybrid search to build advanced search applicationsGetting readyHow to do it...How it works...There’s more...See alsoDeveloping question-answering applications with Generative AIGetting readyHow to do it...How it works...There’s more...See alsoUsing advanced techniques for RAG applicationsGetting readyHow to do it...How it works...There’s more...See also
Chapter 10: Elastic Observability Solution
Technical requirementsInstrumenting your application with Elastic APMGetting readyHow to do it…How it works…There’s more…See alsoSetting up RUMGetting readyHow to do it…How it works…There’s more…See alsoInstrumenting and monitoring with OpenTelemetryGetting readyHow to do it…How it works…There’s more…See alsoMonitoring Kubernetes environments with Elastic AgentGetting readyHow to do it…How it works…There’s more…See alsoManaging synthetics monitoringGetting readyHow to do it…How it works…There’s more…See alsoGaining comprehensive system visibility with Elastic Universal ProfilingGetting readyHow to do it…How it works…There’s more…See alsoDetecting incidents with alerting and machine learningGetting readyHow to do it…How it works…There’s more…See alsoGaining insights with the AI AssistantGetting readyHow to do it…How it works…There’s more…See also
Chapter 11: Managing Access Control
Technical requirementsUsing built-in rolesGetting readyHow to do it…How it works…See alsoDefining custom rolesGetting readyHow to do it…How it works…There’s more…See alsoGranting additional privilegesGetting readyHow to do it…How it works…There’s more…See alsoManaging and securing access to Kibana spacesGetting readyHow to do it…How it works…There’s more…See alsoManaging access with API keysGetting readyHow to do it…How it works…There’s more…See alsoConfiguring single sign-onGetting readyHow to do it…How it works…There’s more…See alsoMapping users and groups to rolesGetting readyHow to do it…How it works…There’s more…
Chapter 12: Elastic Stack Operation
Technical requirementsSetting up an index lifecycle policyGetting readyHow to do it…How it works…There’s more…See alsoOptimizing time series data streams with downsamplingGetting readyHow to do it…How it works…There’s more…See alsoManaging the snapshot lifecycleGetting readyHow to do it…How it works…There’s more…Configuring Elastic Stack components with TerraformGetting readyHow to do it…How it works…There’s more…See alsoEnabling and configuring cross-cluster searchGetting readyHow to do it…How it works…There’s more…See also
Chapter 13: Elastic Stack Monitoring
Technical requirementsSetting up Stack MonitoringGetting readyHow to do it…How it works…There’s more…See alsoBuilding custom visualizations for monitoring dataGetting readyHow to do it…How it works…There’s more…Monitoring cluster health via an APIGetting readyHow to do it…How it works…There’s more…See alsoEnabling audit loggingGetting readyHow to do it…How it works…There’s more…See also
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Elastic Stack 8.x Cookbook

9 Vector Search and Generative AI Integration

In previous chapters, we learned how to build traditional search applications using Query DSL and how to import third-party natural language processing (NLP) models for advanced data processing. In this chapter, we will expand upon these techniques with the latest features of Elastic Stack 8, which include vector search, hybrid search, and integrations with large language models (LLMs), to develop advanced search applications and sophisticated chatbot applications powered by Generative AI. Table 9.1 shows different search strategies supported in Elastic Stack 8 and their use cases:

Search strategies	Use cases
Lexical search (also known as BM25 search; BM stands for best matching) ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781837634293

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Elastic Stack 8.x Cookbook

by Huage Chen, Yazid Akadiri

9

Vector Search and Generative AI Integration

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.