The Definitive Guide to KQL: Using Kusto Query Language for operations, defending, and threat hunting

Book description

Learning KQL is a necessity for system administrators, Azure operators, and security analysts alike, ensuring workloads are monitored to be active, accessible, and secure in the Microsoft Azure cloud platform.

KQL is a powerful query language that helps analyze a large volume of structured, semi structured, and unstructured data. KQL has inbuilt operators and functions that lets a user analyze data to find trends, patterns, anomalies, create forecasting, and machine learning. KQL underpins a variety of Microsoft cloud products - Microsoft Sentinel, Azure Data Explorer, Microsoft 365 Advanced Hunting, Azure Resource Graph, Azure Monitor and more.

This book is designed to be the definitive guide to not only learning KQL but also to using KQL to solve real-world problems. As you learn parts of the language, the authors will show how that can be used to aid with daily operations and security investigations. At the completion of the book, you will have not only learned the language, but also operationalized KQL in your environment.

This topic is important for anyone who manages anything in Azure and any service (AWS, GCP, etc.) being managed through an Azure-based security platform, as well as systems administrators, security consultants, security operations center analysts, and data scientists.

Table of contents

  1. Cover Page
  2. Title Page
  3. Copyright Page
  4. Contents
  5. Table of Contents
  6. Foreword
  7. Dedication
  8. Acknowledgments
  9. About the authors
  10. Introduction
    1. Organization of this book
    2. Who should read this book?
    3. Conventions and features in this book
    4. System requirements
    5. GitHub Repo
    6. Errata, updates & book support
    7. Stay in touch
  11. Chapter 1. Introduction and Fundamentals
    1. Why You Need to Learn KQL
    2. How to Use This Book
    3. Setting Up the Environment
    4. Fundamental Concepts
    5. Searching and Filtering
    6. Data Manipulation
    7. Time Operators
    8. Just Enough User Interface
    9. Miscellaneous Fundamentals
    10. Summary
  12. Chapter 2. Data Aggregation
    1. We Are Dealing with a Lot of Data Here
    2. Obfuscating Results
    3. Distinct and Count
    4. Min, Max, Average, and Sum
    5. Bins, Percentages, and Percentiles
    6. Lists and Sets
    7. Visualizing Data with the Render Operator
    8. Aggregation Functions Usage in Other Operators
    9. Summary
  13. Chapter 3. Unlocking Insights with Advanced KQL Operators
    1. Using KQL variables in KQL
    2. Working with Default Values in Functions
    3. Best Practices for Using Variables in KQL
    4. Uniting Queries with KQL Unions
    5. Union Operator versus Join Operator
    6. Best Practices and Performance Optimization
    7. Joining Data
    8. The externaldata Operator
    9. Query IP Ranges Using KQL
    10. Using the ipv4_is_private() Function
    11. Getting Geolocation from an IP Address Using KQL
    12. Working with Multivalued Strings in KQL
    13. Extracting Multiple Parts from a String
    14. base64_decode_tostring() Function
    15. Working with JSON
    16. Time-Series Analysis
    17. Regular Expressions in KQL
    18. bin() Function
    19. Understanding Functions in Kusto Query Language
    20. Materialize Function
    21. Summary
  14. Chapter 4. Operational Excellence with KQL
    1. Getting Started with KQL
    2. Advanced Hunting with KQL
    3. Common Security Challenges in the Cloud
    4. Hands-on Training: Mastering KQL
    5. Advancing Your KQL Skills
    6. Enabling Diagnostic Settings in Azure
    7. Enabling Diagnostic Settings in Azure Services
    8. Using KQL for Microsoft Intune for Diagnostics and Compliance
    9. Using KQL Queries for Advanced Hunting in Microsoft Defender
    10. Using KQL to Create Powerful Azure Monitor Workbooks
    11. Enhancing Data Management and Efficiency
    12. Best Practices for Optimizing Query Performance
    13. Summary
  15. Chapter 5. KQL for Cybersecurity—Defending and Threat Hunting
    1. Why KQL for security?
    2. Cybersecurity-Focused Operators
    3. User Compromise in Microsoft 365
    4. Phishing Attacks
    5. Firewall Log Parsing
    6. Auditing Security Posture
    7. Microsoft Entra ID (Azure Active Directory) Compromise
    8. Ransomware Tactics, Techniques, and Procedures
    9. Summary
  16. Chapter 6. Advanced KQL Cybersecurity Use Cases and Operators
    1. mv-expand and mv-apply
    2. Joins
    3. let and Nested lets
    4. iff()
    5. case()
    6. coalesce()
    7. More Parsing Operators
    8. regex
    9. Advanced time
    10. Time-series analysis
    11. Geolocation
    12. IP Address Queries
    13. base64_decode_tostring()
    14. toscalar()
    15. evaluate pivot()
    16. Functions
    17. Contributing to the KQL Community
    18. Summary
  17. Keywords
  18. Author Bio

Product information

  • Title: The Definitive Guide to KQL: Using Kusto Query Language for operations, defending, and threat hunting
  • Author(s): Mark Morowczynski, Rod Trent, Matthew Zorich
  • Release date: June 2024
  • Publisher(s): Microsoft Press
  • ISBN: 9780138293482