Apache Kafka for Absolute Beginners

Video description

Explore the Apache Kafka ecosystem and architecture, and learn client API programming in Java

About This Video

  • Get to grips with the Apache Kafka ecosystem and components
  • Delve into Kafka Producer API programming in Java

In Detail

This course is designed to get you up and running with the fundamentals and the working of Apache Kafka with the help of practical examples. You will also delve into client API programming in Java. As you progress, you will not only cover the Apache Kafka stack, but also the architecture of Kafka components and Kafka client APIs (Producers and Consumers). The course will even guide you on how to apply the knowledge you've gained through the course to create efficient Kafka programs in Java.

This course is based on Apache Kafka 2.x. All the source code and examples used in this course on Apache Kafka 2.3 open-source distribution have been tested. Several examples in this course also make use of the Confluent Community Version of Kafka. The course will focus on the Confluent Community Version to explain and demonstrate functionalities that are only available in the Confluent Platform, such as Schema Registry and Avro Serdes.

This course will be making extensive use of IntelliJ IDEA as the preferred development IDE. However, based on your prior experience, you should be able to work with any other IDE designed for Java application development.

Although the course will be using Apache Maven as the preferred build tool, based on your prior experience, you should be able to use any other build tool designed for Java applications.

In addition to this, the course will use Log4j2 to teach you industry-standard log implementation in your applications.

This course is fully example-driven and several examples will be created in the class.

By the end of this course, you will have gained the skills you need to confidently build Kafka programs in Java.

Who this book is for

This course is designed for software engineers, solution architects, and managers looking to implement Kafka and solve real-time stream processing problems. Architects who are not directly involved in Kafka implementation or anyone who is looking to implement Kafka Streams will also find this course helpful. This course is designed for both beginner and intermediate-level Apache Kafka users.

Publisher resources

Download Example Code

Table of contents

  1. Chapter 1 : Kafka Ecosystem - The Big Picture
    1. What is Apache Kafka?
    2. Apache Kafka Core Concepts
    3. Kafka Connect Core Concepts
    4. Kafka Streams Core Concepts
    5. Kafka SQL Core Concepts
    6. When to use what?
  2. Chapter 2 : Apache Kafka- Getting Started
    1. Kafka Quick Start
    2. Installing Single Node Kafka Cluster
    3. Using Command-Line Producer and Consumer
    4. Installing a Multi-Node Kafka Cluster
    5. Using Consumer Groups
    6. Configuring your development IDE
  3. Chapter 3 : Apache Kafka- Storage Architecture
    1. Understanding Kafka Storage Architecture
    2. Kafka Topics and Partitions
    3. Kafka Topic Replication
    4. Partition Leaders and Followers
    5. Kafka Log Segments
    6. Kafka Message Offsets
    7. Kafka Message Index
  4. Chapter 4 : Apache Kafka- Cluster Architecture
    1. Understanding Kafka Cluster
    2. Zookeeper in Kafka
    3. Kafka Cluster Controller
    4. Partition Allocation and Fault Tolerance
    5. Partition Leader Vs Partition Follower
    6. The ISR List - In Sync Replica
    7. Committed Vs Un-Committed Records
    8. Minimum ISR List
  5. Chapter 5 : Apache Producer Internals
    1. Introducing Kafka Producers
    2. Creating your first Kafka Producer
    3. Producer Record
    4. Producer Serializer
    5. Producer Partitioner
    6. Message Timestamp
    7. Producer Message Buffer
    8. Producer IO Thread and Retires
    9. Summarizing Producer Internals
  6. Chapter 6 : Advanced Kafka Producers
    1. Horizontal Vs. Vertical Scalability
    2. Producer Multi-Threading Scenario
    3. Creating Multi-Threaded Kafka Producer
    4. At Least Once Vs. At Most Once
    5. Exactly Once - Producer Idempotence
    6. Transactions in Kafka Producer
  7. Chapter 7 : Types and Serialization
    1. Working with Types and Serialization
    2. Using JSON Schema
    3. Using AVRO Schema
  8. Chapter 8 : Micro Project- Summing up the Producers
    1. POS Simulator using JSON Serialized Invoices
    2. POS Simulator using AVRO Serialized Invoices
  9. Chapter 9 : Kafka Consumers
    1. Introducing Kafka Consumers
    2. Creating Kafka Consume - Transform - Produce Pipeline
    3. Consumer Group and Scalability
    4. Consumer Positions - Current Offset Vs. Committed Offset
  10. Chapter 10 : Where to go Next
    1. Challenges with Kafka Consumers?

Product information

  • Title: Apache Kafka for Absolute Beginners
  • Author(s): Prashant Kumar Pandey
  • Release date: April 2020
  • Publisher(s): Packt Publishing
  • ISBN: 9781800202054