book

Vector Search for Practitioners with Elastic

Name: Vector Search for Practitioners with Elastic
ISBN: 9781805121022

by Bahaaldine Azarmi, Jeff Vestal

November 2023

Intermediate to advanced

240 pages

6h 1m

English

Packt Publishing

Read now

Unlock full access

Vector Search for Practitioners with Elastic
ForewordContributorsAbout the authorsAbout the reviewers
Preface
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesConventions usedGet in touchShare Your ThoughtsDownload a free PDF copy of this book
Part 1:Fundamentals of Vector Search
Chapter 1: Introduction to Vectors and Embeddings
Exploring the roles of supervised and unsupervised learning in vector searchWhat’s an embedding/vector?What challenges are vectors solving?The developer experienceHugging FaceThe market landscape and how it has accelerated the developer experienceUse cases and domains of applicationAI-based searchNamed Entity Recognition (NER)Sentiment analysisText classificationQuestion-answering (QA)Text summarizationHow is Elastic playing a role in this space?A primer on observability and cybersecuritySummary
Chapter 2: Getting Started with Vector Search in Elastic
Search experience in Elastic before vectorsData type and its impact on relevancyThe relevancy modelEvolution of search experienceThe limits of keyword-based searchVector representationThe new vector data type and the vector search query APISparse and dense vectorsAn Elastic Cloud quick startDense vector mappingBrute-force kNN searchkNN searchSummary
Part 2: Advanced Applications and Performance Optimization
Chapter 3: Model Management and Vector Considerations in Elastic
Technical requirementsHugging FaceModel HubDatasetsSpacesElandLoading a Sentence Transformer from Hugging Face into ElasticsearchConfiguring Elasticsearch authenticationLoading a model from the Hugging Face HubDownloading the modelLoading the model into ElasticsearchStarting the modelDeploying the modelGenerating a vector for a queryGenerating vectors in ElasticsearchPlanning for cluster capacity and resourcesCPU and memory requirementsDisk requirementsAnalyze Index Disk Usage APIML node capacityStorage efficiency strategiesReducing dimensionalityQuantizationExcluding dense_vector from _sourceSummary
Chapter 4: Performance Tuning – Working with Data
Deploying an NLP modelLoading a model into ElasticsearchModel deployment configurationsLoad testingRallyRAM estimationTroubleshooting slowdownSummary
Part 3: Specialized Use Cases
Chapter 5: Image Search
Overview of image searchThe evolution of image searchThe mechanism behind image searchThe role of vector similarity searchImage search in practiceVector search with imagesImage vectorizationIndexing image vectors in Elasticsearchk-Nearest Neighbor (kNN) searchChallenges and limitations with image searchMulti-modal models for vector searchIntroduction and rationaleUnderstanding the concept of vector space in multi-modal modelsIntroduction to the OpenAI clip-ViT-B-32-multilingual-v1 modelImplementing vector search for diverse media typesSummary

Chapter 6: Redacting Personal Identifiable Information Using Elasticsearch
Overview of PII and redactionTypes of data that may contain PIIRisks of storing PII in logsHow PII is leaked or lostRedacting PII with NER models and regex patternsNER modelsRegex patternsCombining NER models and regex (or grok) patterns for PII redactionPII redaction pipeline in ElasticsearchGenerating synthetic PIIInstalling the default pipelineExpected resultsExpanding and customizing options for the PII redaction pipeline in ElasticsearchCustomizing the default PII exampleCloning the pipeline to create different versions for different data streamsFine-tuning NER models for particular datasetsLogic for contextual awarenessSummary
Chapter 7: Next Generation of Observability Powered by Vectors
Introduction to observability and its importance in modern software systemsObservability—main pillarsLog analytics and its role in observabilityA new approach—applying vectors and embeddings to log analyticsApproach 1—training or fine-tuning an existing model for logsApproach 2—generating human-understandable descriptions and vectorizing these descriptionsLog vectorizationSynthetic logExpanding logs at write with OpenAISemantic search on our logsBuilding a query using log vectorizationLoading a modelIngest pipelineSemantic searchSummary
Chapter 8: The Power of Vectors and Embedding in Bolstering Cybersecurity
Technical requirementsUnderstanding the importance of email phishing detectionWhat is phishing?Different types of phishing attacksStatistics on the frequency of phishing attacksChallenges in detecting phishing emailsRole of automated detectionAugmenting existing techniques with natural language processingIntroducing ELSERThe role of ELSER in GenAIIntroduction to the Enron email dataset (ham or spam)Seeing ELSER in actionHardware considerationDownloading the ELSER model in ElasticSetting up the index and ingestion pipelineSemantic search with ELSERLimitations of ELSERSummary
Part 4: Innovative Integrations and Future Directions
Chapter 9: Retrieval Augmented Generation with Elastic
Preparing for RAG-enhanced search with ELSER and RRFSemantic search with ELSERA recap of essential considerations for RAGIntegrating ELSER with RRFLanguage models and RAGIn-depth case study—implementing a RAG-enhanced CookBotDataset overview – an introduction to the Allrecipes.com datasetPreparing data for RAG-enhanced searchBuilding the retriever—RRF with ELSERLeveraging the retriever and implementing the generatorSummary
Chapter 10: Building an Elastic Plugin for ChatGPT
Contextual foundationsThe paradigm of dynamic contextDynamic Context Layer plugin vision—architecture and flowBuilding the DCLFetching the latest information from Elastic documentationElevating data with EmbedchainIntegrating with ChatGPT—creating a real-time conversationalistDeploymentSummary
Index
Why subscribe?
Other Books You May EnjoyPackt is searching for authors like youShare Your ThoughtsDownload a free PDF copy of this book

Content preview from Vector Search for Practitioners with Elastic

Part 2: Advanced Applications and Performance Optimization

Dive into the more complex aspects of vector search in Elastic, focusing on model management and performance tuning. These chapters are written for those who are ready to take their knowledge to the next level, offering insights into optimizing search functionalities and enhancing system performance. It’s an in-depth look at the practical applications and fine-tuning required for efficient vector search deployment.

This part has the following chapters:

Chapter 3, Model Management and Vector Considerations in Elastic
Chapter 4, Performance Tuning – Working with Data

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Pretrain Vision and Large Language Models in Python

Publisher Resources

ISBN: 9781805121022

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Vector Search for Practitioners with Elastic

by Bahaaldine Azarmi, Jeff Vestal

Part 2: Advanced Applications and Performance Optimization

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.