Getting Started with Elastic Stack 8.0

Getting Started with Elastic Stack 8.0

by Asjad Athick, Shay Banon
Released March 2022
Publisher(s): Packt Publishing
ISBN: 9781800569492

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Use the Elastic Stack for search, security, and observability-related use cases while working with large amounts of data on-premise and on the cloud

Key Features

  • Learn the core components of the Elastic Stack and how they work together
  • Build search experiences, monitor and observe your environments, and defend your organization from cyber attacks
  • Get to grips with common architecture patterns and best practices for successfully deploying the Elastic Stack

Book Description

The Elastic Stack helps you work with massive volumes of data to power use cases in the search, observability, and security solution areas.

This three-part book starts with an introduction to the Elastic Stack with high-level commentary on the solutions the stack can be leveraged for. The second section focuses on each core component, giving you a detailed understanding of the component and the role it plays. You'll start by working with Elasticsearch to ingest, search, analyze, and store data for your use cases. Next, you'll look at Logstash, Beats, and Elastic Agent as components that can collect, transform, and load data. Later chapters help you use Kibana as an interface to consume Elastic solutions and interact with data on Elasticsearch. The last section explores the three main use cases offered on top of the Elastic Stack. You'll start with a full-text search and look at real-world outcomes powered by search capabilities. Furthermore, you'll learn how the stack can be used to monitor and observe large and complex IT environments. Finally, you'll understand how to detect, prevent, and respond to security threats across your environment. The book ends by highlighting architecture best practices for successful Elastic Stack deployments.

By the end of this book, you'll be able to implement the Elastic Stack and derive value from it.

What you will learn

  • Configure Elasticsearch clusters with different node types for various architecture patterns
  • Ingest different data sources into Elasticsearch using Logstash, Beats, and Elastic Agent
  • Build use cases on Kibana including data visualizations, dashboards, machine learning jobs, and alerts
  • Design powerful search experiences on top of your data using the Elastic Stack
  • Secure your organization and learn how the Elastic SIEM and Endpoint Security capabilities can help
  • Explore common architectural considerations for accommodating more complex requirements

Who this book is for

Developers and solutions architects looking to get hands-on experience with search, security, and observability-related use cases on the Elastic Stack will find this book useful. This book will also help tech leads and product owners looking to understand the value and outcomes they can derive for their organizations using Elastic technology. No prior knowledge of the Elastic Stack is required.

Table of contents

  1. Getting Started with Elastic Stack 8.0
  2. Foreword
  3. Contributors
  4. About the author
  5. About the reviewers
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Download the color images
    6. Conventions used
    7. Get in touch
    8. Share Your Thoughts
  7. Section 1: Core Components
  8. Chapter 1: Introduction to the Elastic Stack
    1. An overview of the Elastic Stack
      1. The evolution of the Elastic Stack
    2. A note about licensing
    3. What is Elasticsearch?
      1. When to use Elasticsearch
      2. Architectural characteristics of Elasticsearch
      3. When Elasticsearch may not be the right tool
    4. Introducing Kibana
    5. Collecting and ingesting data
      1. Collecting data from across your environment using Beats
      2. Centralized extraction and transformation and loading your data with Logstash
      3. Deciding between using Beats and Logstash
    6. Running the Elastic Stack
      1. Standalone deployments
      2. Elastic Cloud
    7. Solutions built on the stack
      1. Enterprise Search
      2. Security
      3. Observability
    8. Summary
  9. Chapter 2: Installing and Running the Elastic Stack
    1. Technical requirements
    2. Manual installation of the stack
      1. Installing on Linux
    3. Automating the installation
      1. Using Ansible for automation
    4. Using Elastic Cloud Enterprise (ECE) for orchestration
      1. ECE architecture
      2. Proxies
      3. ECE installation size
      4. Installing ECE
      5. Creating your deployment on ECE
    5. Running on Kubernetes
    6. Configuration of your lab environment
    7. Summary
  10. Section 2: Working with the Elastic Stack
  11. Chapter 3: Indexing and Searching for Data
    1. Technical requirements
    2. Understanding the internals of an Elasticsearch index
      1. Inside an index
    3. Elasticsearch nodes
      1. Master-eligible nodes
      2. Voting-only nodes
      3. Data nodes
      4. Ingest nodes
      5. Coordinator nodes
      6. Machine learning nodes
      7. Elasticsearch clusters
    4. Searching for data
      1. Indexing sample logs
      2. Running queries on your data
    5. Summary
  12. Chapter 4: Leveraging Insights and Managing Data on Elasticsearch
    1. Technical requirements
    2. Getting insights from data using aggregations
    3. Managing the life cycle of time series data
      1. The usefulness of data over time
      2. Index Lifecycle Management
      3. Using data streams to manage time series data
    4. Manipulating incoming data with ingest pipelines
      1. Common use cases for ingest pipelines
    5. Responding to changing data with Watcher
      1. Getting started with Watcher
      2. Common use cases for Watcher
    6. Summary
  13. Chapter 5: Running Machine Learning Jobs on Elasticsearch
    1. Technical requirements
    2. The value of running machine learning on Elasticsearch
    3. Preparing data for machine learning jobs
      1. Machine learning concepts
    4. Looking for anomalies in time series data
      1. Looking for anomalous event rates in application logs
      2. Looking for anomalous data transfer volumes
      3. Comparing the behavior of source IP addresses against the population
    5. Running classification on data
      1. Predicting maliciously crafted requests using classification
    6. Inferring against incoming data using machine learning
    7. Summary
  14. Chapter 6: Collecting and Shipping Data with Beats
    1. Technical requirements
    2. Introduction to Beats agents
    3. Collecting logs using Filebeat
    4. Using Metricbeat to monitor system and application metrics
    5. Monitoring operating system audit data using Auditbeat
    6. Monitoring the uptime and availability of services using Heartbeat
    7. Collecting network traffic data using Packetbeat
    8. Summary
  15. Chapter 7: Using Logstash to Extract, Transform, and Load Data
    1. Technical requirements
    2. Introduction to Logstash
      1. Understanding how Logstash works
      2. Configuring your Logstash instance
      3. Running your first pipeline
    3. Looking at pipelines for real-world data-processing scenarios
      1. Loading data from CSV files into Elasticsearch
      2. Parsing Syslog data sources
      3. Enriching events with contextual data
      4. Aggregating event streams into a single event
      5. Processing custom logs collected by Filebeat using Logstash
    4. Summary
  16. Chapter 8: Interacting with Your Data on Kibana
    1. Technical requirements
    2. Getting up and running on Kibana
      1. Solutions in Kibana
      2. Kibana data views
    3. Visualizing data with dashboards
    4. Creating data-driven presentations with Canvas
    5. Working with geospatial datasets using Maps
    6. Responding to changes in data with alerting
      1. The anatomy of an alert
      2. Creating alerting rules
    7. Summary
  17. Chapter 9: Managing Data Onboarding with Elastic Agent
    1. Technical requirements
    2. Tackling the challenges in onboarding new data sources
      1. Unified data collection using a single agent
    3. Managing Elastic Agent at scale with Fleet
      1. Agent policies and integrations
    4. Setting up your environment
      1. Preparing your Elasticsearch deployment for Fleet
      2. Setting up Fleet Server to manage your agents
      3. Collecting data from your web server using Elastic Agent
      4. Using integrations to collect data
    5. Summary
  18. Section 3: Building Solutions with the Elastic Stack
  19. Chapter 10: Building Search Experiences Using the Elastic Stack
    1. Technical requirements
    2. An introduction to full-text searching
      1. Analyzing text for a search
      2. Running searches
    3. Implementing features to improve the search experience
      1. Autocompleting search queries
      2. Suggesting search terms for queries
      3. Using filters to narrow down search results
      4. Paginating large result sets
      5. Ordering search results
      6. Putting it all together to implement recipe search functionality
    4. Summary
  20. Chapter 11: Observing Applications and Infrastructure Using the Elastic Stack
    1. Technical requirements
    2. An introduction to observability
      1. Metrics
      2. Logs
      3. Traces
      4. Synthetic and real user monitoring
    3. Observing your environment
      1. Infrastructure-level visibility
      2. Platform-level visibility
      3. Host- and operating system-level visibility
      4. Monitoring your software workloads
      5. Leveraging out-of-the-box content for observability data
    4. Instrumenting your application performance
      1. Configuring APM to instrument your code
    5. Summary
  21. Chapter 12: Security Threat Detection and Response Using the Elastic Stack
    1. Technical requirements
    2. Building security capability to protect your organization
      1. Confidentiality
      2. Integrity
      3. Availability
    3. Building a SIEM for your SOC
      1. Collecting data from a range of hosts and source systems
      2. Monitoring and detecting security threats in near real time
      3. Allowing analysts to work and investigate collaboratively
      4. Applying threat intelligence and data enrichment to contextualize your alerts
      5. Enabling teams to hunt for adversarial behavior in the environment
      6. Providing alerting, integrations, and response actions
      7. Easily scaling with data volumes over suitable data retention periods
    4. Leveraging endpoint detection and response in your SOC
      1. Malware
      2. Ransomware
      3. Memory threats
      4. Malicious behavior
    5. Summary
  22. Chapter 13: Architecting Workloads on the Elastic Stack
    1. Architecting workloads on Elastic Stack
      1. Designing for high availability
      2. Scaling your workloads with your data
      3. Recovering your workloads from disaster
      4. Securing your workloads on Elastic Stack
    2. Architectures to handle complex requirements
      1. Federating searches across different Elasticsearch deployments
      2. Replicating data between your Elasticsearch deployments
      3. Using tiered data architectures for your deployment
    3. Implementing successful deployments of the Elastic Stack
    4. Summary
    5. Why subscribe?
  23. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts

Product information

  • Title: Getting Started with Elastic Stack 8.0
  • Author(s): Asjad Athick, Shay Banon
  • Release date: March 2022
  • Publisher(s): Packt Publishing
  • ISBN: 9781800569492