O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Elasticsearch 5 and Elastic Stack - In Depth and Hands On!

Video Description

Search, analyze, and visualize big data on a cluster with Elasticsearch, Logstash, Beats, Kibana, and more

About This Video

  • Install and configure Elasticsearch on a cluster
  • Create search indices and mappings
  • Search full-text and structured data in several different ways
  • Import data into Elasticsearch using several different techniques
  • Integrate Elasticsearch with other systems, such as Spark, Kafka, relational databases, S3, and more
  • Aggregate structured data using buckets and metrics
  • Use Logstash and the "ELK stack" to import streaming log data into Elasticsearch
  • Use Filebeats and the Elastic Stack to import streaming data at scale
  • Analyze and visualize data in Elasticsearch using Kibana
  • Manage operations on production Elasticsearch clusters
  • Use cloud-based solutions including Amazon's Elasticsearch Service and Elastic Cloud

In Detail

Elasticsearch is a powerful tool not only for powering search on big websites, but also for analyzing big data sets in a matter of milliseconds! It's an increasingly popular technology, and a valuable skill to have in today's job market. We'll cover setting up search indices on an Elasticsearch cluster, and querying that data in many different ways. Fuzzy searches, partial matches, search-as-you-type, pagination, sorting - you name it. And it's not just theory, every lesson has hands-on examples where you'll practice each skill using a virtual machine running Elasticsearch on your own PC. We cover, in depth, the often-overlooked problem of importing data into an Elasticsearch index. Whether it's via raw RESTful queries, scripts using Elasticsearch API's, or integration with other "big data" systems like Spark and Kafka - you'll see many ways to get Elasticsearch started from large, existing data sets at scale. We'll also stream data into Elasticsearch using Logstash and Filebeat - commonly referred to as the "ELK Stack" (Elasticsearch / Logstash / Kibana) or the "Elastic Stack". Elasticsearch isn't just for search anymore - it has powerful aggregation capabilities for structured data. We'll bucket and analyze data using Elasticsearch, and visualize it using the Elastic Stack's web UI, Kibana. Elasticsearch is positioning itself to be a much faster alternative to Hadoop, Spark, and Flink for many common data analysis requirements. It's an important tool to understand, and it's easy to use! Dive in with me and I'll show you what it's all about.

Table of Contents

  1. Chapter 1 : Installing and Understanding Elasticsearch
    1. Introduction, and Installing Elasticsearch 00:17:12
    2. Elasticsearch Overview 00:05:44
    3. Using Elasticsearch 00:09:02
    4. Elasticsearch Architecture 00:06:47
    5. Quiz: Elasticsearch Concepts and Architecture 00:04:24
  2. Chapter 2 : Mapping and Indexing Data
    1. Connecting to your Cluster 00:06:54
    2. Getting to Know the Movielens Data Set 00:04:17
    3. Create a Mapping for MovieLens 00:15:01
    4. Import a Single Movie via JSON / REST 00:05:00
    5. Insert Many Movies at Once 00:05:33
    6. Updating Data in Elasticsearch 00:06:13
    7. Deleting Data in Elasticsearch 00:02:43
    8. [Exercise] Insert, Update, and Delete a Fictitious Movie 00:04:27
    9. Dealing With Concurrency 00:08:45
    10. Using Analyzers and Tokenizers 00:12:35
    11. Data Modeling with Elasticsearch 00:13:37
  3. Chapter 3 : Searching with Elasticsearch
    1. Using Query-String Search 00:09:13
    2. Using JSON Search 00:09:54
    3. Full-Text vs. Phrase Search 00:06:04
    4. [Exercise] Search for New Star Wars Films Two Different Ways 00:03:50
    5. Pagination 00:06:14
    6. Sorting 00:07:24
    7. Using Filters 00:04:24
    8. [Exercise] Search for Science Fiction Movies Before 1960, Sorted by Title 00:03:10
    9. Fuzzy Queries 00:06:16
    10. Partial Matching 00:05:26
    11. N-Grams, and Search as You Type 00:13:13
  4. Chapter 4 : Importing Data Into Your Index - Big or Small
    1. Importing Data from Scripts 00:15:10
    2. [Exercise] Import Movie Tags Into a New Index with a Python Script 00:03:33
    3. Logstash Overview 00:04:32
    4. Installing Logstash 00:07:49
    5. Importing Apache Access Logs with Logstash 00:04:52
    6. Importing Data from MySQL using Logstash 00:14:34
    7. Importing Data from AWS S3 using Logstash 00:08:14
    8. Integrating Kafka with Elasticsearch 00:09:01
    9. Integrating Spark and Hadoop with Elasticsearch 00:12:22
    10. [Exercise] Import Movie Ratings from Spark to Elasticsearch 00:09:21
  5. Chapter 5 : Aggregation
    1. Buckets and Metrics 00:12:04
    2. Histograms 00:07:37
    3. Aggregating Time Series Data 00:07:16
    4. [Exericse] When Did my Site Go Down? 00:04:06
    5. Nested Aggregations 00:13:52
  6. Chapter 6 : Using Kibana
    1. Installing Kibana 00:06:06
    2. Analyzing Shakespeare with Kibana 00:08:45
    3. [Exercise] Find the Shakespeare Plays with the Most Lines 00:03:14
  7. Chapter 7 : Analyzing Log Data with the Elastic Stack
    1. The ELK Stack and Elastic Stack 00:04:13
    2. Install, Configure, and Use FileBeat 00:07:33
    3. Analyzing Server Logs with Kibana 00:07:05
    4. [Exercise] Narrow Down the Source of 404 Errors 00:04:19
  8. Chapter 8 : Elasticsearch Operations
    1. How Many Shards Should I Use? 00:05:49
    2. Scaling with New Indices 00:07:41
    3. Choosing Your Hardware 00:03:04
    4. Heap Sizing 00:03:16
    5. Monitoring with X-Pack 00:11:56
    6. Practicing Failover 00:12:25
    7. Snapshots 00:09:30
    8. Rolling Restarts 00:08:06
  9. Chapter 9 : Elasticsearch in the Cloud
    1. Using Amazon Elasticsearch Service 00:10:19
    2. Integrating Amazon ES with Logstash 00:14:10
    3. Using Elastic Cloud 00:09:36
  10. Chapter 10 : You Made It!
    1. I Made It! Now What? 00:03:46