book

Elasticsearch 5.x Cookbook - Third Edition

Name: Elasticsearch 5.x Cookbook - Third Edition
Author: Alberto Paro
ISBN: 9781786465580

by Alberto Paro

February 2017

Intermediate to advanced

696 pages

12h 24m

English

Packt Publishing

Read now

Unlock full access

Elasticsearch 5.x Cookbook Third Edition
Credits
About the Author
About the Reviewer
www.PacktPub.com
eBooks, discount offers, and moreWhy subscribe?
Customer Feedback
Dedication
Preface
What this book covers
What you need for this book

Who this book is for
Sections
Getting readyHow to do it…How it works…There's more…See also
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Getting Started
Introduction
Understanding node and cluster
Getting readyHow it work...There's more...See also
Understanding node services
Getting readyHow it works...
Managing your data
Getting readyHow it works...There's more...Best practicesSee also
Understanding cluster, replication, and sharding
Getting readyHow it works...Best practiceThere's more...Solving the yellow statusSolving the red statusSee also
Communicating with Elasticsearch
Getting readyHow it works...
Using the HTTP protocol
Getting readyHow to do it...How it works...There's more...
Using the native protocol
Getting readyHow to do it...How it works...There's more...See also
2. Downloading and Setup
Introduction
Downloading and installing Elasticsearch
Getting readyHow to do it...How it works...There's more...See also
Setting up networking
Getting readyHow to do it...How it works...See also
Setting up a node
Getting readyHow to do it...How it works...There's more...See also
Setting up for Linux systems
Getting readyHow to do it...How it works...
Setting up different node types
Getting readyHow to do it...How it works...
Setting up a client node
Getting readyHow to do it...How it works...
Setting up an ingestion node
Getting readyHow to do it...How it works...
Installing plugins in Elasticsearch
Getting readyHow to do it...How it works...There's more...See also
Installing plugins manually
Getting readyHow to do it...How it works...
Removing a plugin
Getting readyHow to do it...How it works...
Changing logging settings
Getting readyHow to do it...How it works...
Setting up a node via Docker
Getting readyHow to do it...How it works...There's more...See also
3. Managing Mappings
Introduction
Using explicit mapping creation
Getting readyHow to do it...How it works...There's more...See also
Mapping base types
Getting readyHow to do it...How it works...There's more...See also
Mapping arrays
Getting readyHow to do it...How it works...
Mapping an object
Getting readyHow to do it...How it works...See also
Mapping a document
Getting readyHow to do it...How it works...See also
Using dynamic templates in document mapping
Getting readyHow to do it...How it works...There's more...See also
Managing nested objects
Getting readyHow to do it...How it works...There's more...See also
Managing child document
Getting readyHow to do it...How it works...There's more...See also
Adding a field with multiple mapping
Getting readyHow to do it...How it works...There's more...See also
Mapping a GeoPoint field
Getting readyHow to do it...How it works...There's more...
Mapping a GeoShape field
Getting readyHow to do itHow it works...See also
Mapping an IP field
Getting readyHow to do it...How it works...
Mapping an attachment field
Getting readyHow to do it...How it works...There's more...See also
Adding metadata to a mapping
Getting readyHow to do it...How it works...
Specifying a different analyzer
Getting readyHow to do it...How it works...See also
Mapping a completion field
Getting readyHow to do it...How it works...See also
4. Basic Operations
Introduction
Creating an index
Getting readyHow to do it...How it works...There's more...See also
Deleting an index
Getting readyHow to do it...How it works...See also
Opening/closing an index
Getting readyHow to do it...How it works...See also
Putting a mapping in an index
Getting readyHow to do it...How it works...There's more...See also
Getting a mapping
Getting readyHow to do it...How it works...See also
Reindexing an index
Getting readyHow to do it...How it works...
See also
Refreshing an index
Getting readyHow to do it...How it works...See also
Flushing an index
Getting readyHow to do it...How it works...See also
ForceMerge an index
Getting readyHow to do it...How it works...There's more...See also
Shrinking an index
Getting readyHow to do it...How it works...There's more...See also
Checking if an index or type exists
Getting readyHow to do it...How it works...
Managing index settings
Getting readyHow to do it...How it works...There's more...See also
Using index aliases
Getting readyHow to do it...How it works...There's more...
Rollover an index
Getting readyHow to do it…How it works...See also
Indexing a document
Getting readyHow to do it...How it works...There's more...See also
Getting a document
Getting readyHow to do it...How it works...There is more...See also
Deleting a document
Getting readyHow to do it...How it works...See also
Updating a document
Getting readyHow to do it...How it works...See also
Speeding up atomic operations (bulk operations)
Getting readyHow to do it...How it works...
Speeding up GET operations (multi GET)
Getting readyHow to do it...How it works...See also...
5. Search
Introduction
Executing a search
Getting readyHow to do it...How it works...There's more...See also
Sorting results
Getting readyHow to do it...How it works...There's more...See also
Highlighting results
Getting readyHow to do it...How it works…See also
Executing a scrolling query
Getting readyHow to do it...How it works...There's more...See also
Using the search_after functionality
Getting readyHow to do it...How it works...See also
Returning inner hits in results
Getting readyHow to do it...How it works...See also
Suggesting a correct query
Getting readyHow to do it...How it works...See also
Counting matched results
Getting readyHow to do it...How it works...There's more...See also
Explaining a query
Getting readyHow to do it...How it works...
Query profiling
Getting readyHow to do it...How it works...
Deleting by query
Getting readyHow to do it...How it works...There's more...See also
Updating by query
Getting readyHow to do it...How it works...There's more...See also
Matching all the documents
Getting readyHow to do it...How it works...See also
Using a boolean query
Getting readyHow to do it...How it works...
6. Text and Numeric Queries
Introduction
Using a term query
Getting readyHow to do it...How it works...There's more...
Using a terms query
Getting readyHow to do it...How it works...There's more...See also
Using a prefix query
Getting readyHow to do it...How it works...There's more...See also
Using a wildcard query
Getting readyHow to do it...How it works...See also
Using a regexp query
Getting readyHow to do it...How it works...See also
Using span queries
Getting readyHow to do it...How it works...See also
Using a match query
Getting readyHow to do it...How it works...See also
Using a query string query
Getting readyHow to do it...How it works...There's more...See also
Using a simple query string query
Getting readyHow to do it...How it works...See also
Using the range query
Getting readyHow to do it...How it works...There's more...
The common terms query
Getting readyHow to do it...How it works...See also
Using IDs query
Getting readyHow to do it...How it works...See also
Using the function score query
Getting readyHow to do it...How it works...See also
Using the exists query
Getting readyHow to do it...How it works...
Using the template query
Getting readyHow to do it...How it works...There's more...See also
7. Relationships and Geo Queries
Introduction
Using the has_child query
Getting readyHow to do it...How it works...There's more...See also
Using the has_parent query
Getting readyHow to do it...How it works...See also
Using nested queries
Getting readyHow to do it...How it works...See also
Using the geo_bounding_box query
Getting readyHow to do it...How it works...See also
Using the geo_polygon query
Getting readyHow to do it...How it works...See also
Using the geo_distance query
Getting readyHow to do it...How it works...See also
Using the geo_distance_range query
Getting readyHow to do it...How it works...See also
8. Aggregations
Introduction
Executing an aggregation
Getting readyHow to do it...How it works...See also
Executing stats aggregations
Getting readyHow to do it...How it works...See also
Executing terms aggregation
Getting readyHow to do it...How it works...There's more...See also
Executing significant terms aggregation
Getting readyHow to do it...How it works...
Executing range aggregations
Getting readyHow to do it...How it works...There's more...See also
Executing histogram aggregations
Getting readyHow to do it...How it works...There's more...See also
Executing date histogram aggregations
Getting readyHow to do it...How it works...See also
Executing filter aggregations
Getting readyHow to do it...How it works...There's more...See also
Executing filters aggregations
Getting readyHow to do it...How it works...
Executing global aggregations
Getting readyHow to do it...How it works...
Executing geo distance aggregations
Getting readyHow to do it...How it works...See also
Executing children aggregations
Getting readyHow to do it...How it works...
Executing nested aggregations
Getting readyHow to do it...How it works...There's more...
Executing top hit aggregations
Getting readyHow to do it...How it works...See also
Executing a matrix stats aggregation
Getting readyHow to do it...How it works...
Executing geo bounds aggregations
Getting readyHow to do it...How it works...See also
Executing geo centroid aggregations
Getting readyHow to do it...How it works...See also
9. Scripting
Introduction
Painless scripting
Getting readyHow to do it...How it works...There's moreSee also
Installing additional script plugins
Getting readyHow to do it...How it works...There's more...
Managing scripts
Getting readyHow to do it...How it works...There's more...See also
Sorting data using scripts
Getting readyHow to do it...How it works...There's more...
Computing return fields with scripting
Getting readyHow to do it...How it works...See also
Filtering a search via scripting
Getting readyHow to do it...How it works...There's more...See also
Using scripting in aggregations
Getting readyHow to do it...How it works...
Updating a document using scripts
Getting readyHow to do it...How it works...There's more...
Reindexing with a script
Getting readyHow to do it...How it works...
10. Managing Clusters and Nodes
11. Backup and Restore
Introduction
Managing repositories
Getting readyHow to do it...How it works...There's more...See also
Executing a snapshot
Getting readyHow to do it...How it works...There's more...
Restoring a snapshot
Getting readyHow to do it...How it works...
Setting up a NFS share for backup
Getting readyHow to do it...How it works...
Reindexing from a remote cluster
Getting readyHow to do it...How it works...See also
12. User Interfaces
Introduction
Installing and using Cerebro
Getting readyHow to do it...How it works...There's more...
Installing Kibana and X-Pack
Getting readyHow to do it...How it works...
Managing Kibana dashboards
Getting readyHow to do it...How it works...
Monitoring with Kibana
Getting readyHow to do it...How it works...See also
Using Kibana dev-console
Getting readyHow to do it...How it works...There's more...
Visualizing data with Kibana
Getting readyHow to do it...How it works...
Installing Kibana plugins
Getting readyHow to do it...How it works...
Generating graph with Kibana
Getting readyHow to do it...How it works...
13. Ingest
Introduction
Pipeline definition
Getting readyHow to do it...How it works...There's more...
Put an ingest pipeline
Getting readyHow to do it...How it works...
Get an ingest pipeline
Getting readyHow to do it...How it works...There's more...
Delete an ingest pipeline
Getting readyHow to do it...How it works...
Simulate an ingest pipeline
Getting readyHow to do it...How it works...There's more...
Built-in processors
Getting readyHow to do it...How it works...See also
Grok processor
Getting readyHow to do it...How it works...See also
Using the ingest attachment plugin
Getting readyHow to do it...How it works...
Using the ingest GeoIP plugin
Getting readyHow to do it...How it works...See also
14. Java Integration
Introduction
Creating a standard Java HTTP client
Getting readyHow to do it...How it works...See also
Creating an HTTP Elasticsearch client
Getting readyHow to do it...How it works...See also
Creating a native client
Getting readyHow to do it...How it works...There's more...See also
Managing indices with the native client
Getting readyHow to do it...How it works...See also
Managing mappings
Getting readyHow to do it...How it works...There's more...See also
Managing documents
Getting readyHow to do it...How it works...See also
Managing bulk actions
Getting readyHow to do it...How it works...
Building a query
Getting readyHow to do it...How it works...There's more...
Executing a standard search
Getting readyHow to do it...How it works...See also
Executing a search with aggregations
Getting readyHow to do it...How it works...See also
Executing a scroll search
Getting readyHow to do it...How it works...See also
15. Scala Integration
Introduction
Creating a client in Scala
Getting readyHow to do it...How it works...See also
Managing indices
Getting readyHow to do it...How it works...See also
Managing mappings
Getting readyHow to do it...How it works...See also
Managing documents
Getting readyHow to do it...How it works...There's more...See also
Executing a standard search
Getting readyHow to do it...How it works...See also
Executing a search with aggregations
Getting readyHow to do it...How it works...See also
16. Python Integration
Introduction
Creating a client
Getting readyHow to do it...How it works…See also
Managing indices
Getting readyHow to do it…How it works…There's more…See also
Managing mappings include the mapping
Getting readyHow to do it…How it works…See also
Managing documents
Getting readyHow to do it…How it works…See also
Executing a standard search
Getting readyHow to do it…How it works…See also
Executing a search with aggregations
Getting readyHow to do it…How it works…See also
17. Plugin Development
Introduction
Creating a plugin
Getting readyHow to do it...How it works...There's more...
Creating an analyzer plugin
Getting readyHow to do it...How it works...There's more...
Creating a REST plugin
Getting readyHow to do it...How it works...See also
Creating a cluster action
Getting readyHow to do it...How it works...See also
Creating an ingest plugin
Getting readyHow to do it...How it works...
18. Big Data Integration
Introduction
Installing Apache Spark
Getting readyHow to do it...How it works...There's more...
Indexing data via Apache Spark
Getting readyHow to do it...How it works...See also
Indexing data with meta via Apache Spark
Getting readyHow to do it...How it works...There's more...
Reading data with Apache Spark
Getting readyHow to do it...How it works...
Reading data using SparkSQL
Getting readyHow to do it...How it works...
Indexing data with Apache Pig
Getting readyHow to do it...How it works...

Content preview from Elasticsearch 5.x Cookbook - Third Edition

Put an ingest pipeline

The power of the pipeline definition is the ability for to be updated and created without a node restart (compared to Logstash). The definition is stored in a cluster state via the put pipeline API.

After having defined a pipeline, we need to provide it to the Elasticsearch cluster.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 2, Downloading and Setup.

To execute curl via the command line, you need to install curl for your operative system.

How to do it...

To store or update an ingestion pipeline in Elasticsearch, we will perform the following steps:

We can store the ingest pipeline via a PUT call:

 curl -XPUT 'http://127.0.0.1:9200/_ingest/pipeline/add-user- ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Elasticsearch Server - Third Edition - Third Edition

Publisher Resources

ISBN: 9781786465580

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills