O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Apache Kafka Series - Learn Apache Kafka for Beginners

Video Description

Tutorial: Learn the Apache Kafka ecosystem, core concepts, operations, Kafka API, and build your own producers and consumers

About This Video

  • Learn the Apache Kafka ecosystem architecture
  • Learn the Kafka core concepts: topics, partitions, brokers, replicas, producers, consumers
  • Launch your own Kafka cluster in no time using Docker – Windows / MacOS X / Linux
  • Learn and practice using the Kafka command line tools
  • Learn and code consumers and producers using the Java API
  • Learn advanced Kafka topic configurations
  • Pointers for integration between Kafka and other frameworks (Spark, NiFi, Akka, Scala).

In Detail

Apache Kafka has become the leading data-streaming enterprise technology. Kafka is used in production by over 2000 companies like Netflix, Airbnb, Uber and LinkedIn. To learn Kafka easily, step-by-step, you have come to the right place! Apache Kafka and its ecosystem: In this section, we will learn about the Apache Kafka ecosystem, and see how some target architectures may look. This high-level section helps you to set context around Kafka! Apache Kafka Core concepts: In this section, we will learn about all the fundamental concepts of Kafka like topics, partitions, replication, brokers, producers, consumers, consumer groups, Zookeeper, delivery semantics, and more! Docker and Kafka Setup: In this section, we will learn how to install Docker on your machine and get started with Apache Kafka, in the simplest way possible. Apache Kafka Hands-on Practice: In this section, we will gain some practical experience by learning how the various command lines tool work, as well as how to use the Kafka Topics UI, and create your very first producer and consumer in Java. Code Examples - Libraries Integrations: In this section, we will learn about some more advanced code examples, and understand where to find the libraries to integrate with frameworks such as Spark, Spark Streaming, Akka Streams, Scala, Actors, Apache NiFi. Advanced Topic Configuration: In this section, we will understand the main configurations for your topics, learn about log compaction, and understand exactly what your partitions are made of!

Downloading the example code for this course: You can download the example code files for all Packt video courses you have purchased from your account at http://www.PacktPub.com. If you purchased this course elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. Chapter 1 : Introduction to Kafka and the ecosystem
    1. Course Objectives 00:01:16
    2. Pre-requisites before getting started 00:01:17
    3. What is Apache Kafka? 00:03:03
    4. Kafka and the Kafka ecosystem: high level overview 00:08:07
    5. Kafka Architecture within the enterprise 00:01:19
    6. How to ask questions in the Q&A 00:02:26
  2. Chapter 2 : Apache Kafka: Core Concepts
    1. Topics and Partitions 00:04:32
    2. Brokers and Data Replication 00:06:09
    3. Producers 00:04:43
    4. Consumers and Consumer Groups 00:04:57
    5. Zookeeper 00:02:05
    6. Kafka Guarantees, Message Ordering 00:02:17
    7. Delivery Semantics: At least once, at most once, exactly once 00:03:58
  3. Chapter 3 : Apache Kafka: Install Docker and Start Kafka!
    1. Pre-requisite: Docker on Mac (recent versions) 00:03:54
    2. Pre-requisite: Docker Toolbox on Mac (older versions) 00:05:17
    3. Pre-requisite: Docker on Linux (Ubuntu as an example) 00:04:56
    4. Pre-requisite: Docker on Windows 10 64bit 00:03:43
    5. Pre-requisite: Docker Toolbox on Windows (other versions) 00:05:37
    6. Starting Kafka! 00:06:09
  4. Chapter 4 : Apache Kafka: Hands-on Practice
    1. Topic operations: create, list, delete, describe 00:10:49
    2. Publishing data to a topic using the console producer 00:06:44
    3. Consuming data from a topic using the console consumer 00:09:19
    4. Kafka Topics UI 00:03:58
    5. Writing your own producer 00:12:49
    6. Writing your own consumer 00:12:30
  5. Chapter 5 : Code Examples - Library Integrations
    1. Kafka Producer and Consumer Examples 00:11:40
    2. Spark & Spark Streaming Integration 00:12:57
    3. Akka Streams Integration 00:13:37
    4. Scala Library & Actor Model Integration 00:04:58
    5. Apache NiFi Integration 00:14:43
  6. Chapter 6 : Advanced: Topic Configuration
    1. Why should I care about Topic Configuration? 00:02:32
    2. Hands on: Changing Topic Configurations 00:04:26
    3. Partitions Count & Replication Factor 00:07:20
    4. Segments and Indexes 00:06:20
    5. Log Cleanup Policy: Overview 00:03:54
    6. Log Cleanup Policy: "delete" based on Time and Bytes 00:03:29
    7. Log Cleanup Policy: Log Compaction – Theory 00:08:04
    8. Hands On: Log Compaction 00:08:25
    9. Log Compression 00:02:26
  7. Chapter 7 : Next steps!
    1. What's next? 00:02:52