Kafka Streams with Spring Cloud Stream

Kafka Streams with Spring Cloud Stream

by Prashant Kumar Pandey
Released July 2021
Publisher(s): Packt Publishing
ISBN: 9781801811422

Watch it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Video description

Kafka Streams with Spring Cloud Streams will help you understand stream processing in general and apply it to Kafka Streams Programming using Spring Boot.

This course uses the Kafka Streams library compatible with Spring Cloud 2020. All the source code and examples used in this course have been tested by the author on Confluent Platform 6.0.0, which is compatible with Apache Kafka 2.6 open-source distribution.

This is a fully example-driven course, and you will be working with multiple examples during the entire session. We will be making extensive use of IntelliJ IDEA as the preferred development IDE and Apache Maven and Gradle as the preferred build tool. However, based on your prior experience, you should be able to work with any other IDE designed for Spring application development and any other build tool designed for Java applications.

This course also makes use of Log4J2 to teach you industry-standard log implementation in your application. We will be using JUnit5, which is the latest version of JUnit, to implement unit test cases.

Working examples and exercises are the most critical tool to sharpen your skills. This course consists of some programming assignments as and when appropriate. These exercises will help you validate and check your concepts and apply your learning to solve programming problems.

What You Will Learn

  • Designing, developing, and testing stream processing applications
  • Kafka Streams binder implementation for Spring cloud streams
  • Working with JSON, AVRO, and other custom serializations
  • Spring cloud streams and Kafka Streams architecture
  • Kafka Streams DSL and programming with Kafka Streams API
  • Unit-testing Kafka Streams application

Audience

Kafka Streams with Spring Cloud Streams is designed for software engineers willing to develop a stream processing application using the Kafka Streams library and Spring Boot. This course has also been created for data architects and data engineers responsible for designing and building the organization's data-centric infrastructure. Another group of people is the managers and architects who do not directly work with Kafka implementation, but they work with the people who implement Kafka Streams at the ground level.

About The Author

Prashant Kumar Pandey: Prashant Kumar Pandey is passionate about helping people learn and grow in their careers by bridging the gap between their existing and required skills. In his journey to fulfill this mission, he is authoring books, publishing technical articles, and creating training videos to help IT professionals and students succeed in the industry. He is also the founder, lead author, and chief editor of the Learning Journal portal that has been providing various skill development courses, training sessions, and technical articles since 2018.

Table of contents

  1. Chapter 1 : Before You Begin
    1. About the Course
    2. Course Prerequisite
    3. Setting Up Your Environment
    4. Creating Your Starter Project
  2. Chapter 2 : Environment Setup on Window 10 Machine
    1. Installing Confluent Kafka - Windows WSL
    2. Creating Your First Kafka Streams Project – Windows.
  3. Chapter 3 : Environment Setup on Mac Machine
    1. Installing Confluent Kafka - Mac
    2. Creating Your First Kafka Streams Project – Mac
  4. Chapter 4 : Understanding the Technology Stack
    1. Understanding Kafka Support in Spring
    2. Introduction to Spring Cloud Streams
    3. Introduction to Kafka Streams
  5. Chapter 5 : Producing Data to Kafka
    1. Simple RESTful Kafka Producer
    2. Creating Retail POS Simulator
    3. Producing JSON Messages
    4. Producing AVRO Messages
  6. Chapter 6 : Processing Kafka Streams
    1. Real-Time Stream Processing – Requirement
    2. Processing JSON Message Stream
    3. Real-Life Serialization Scenarios
    4. Processing AVRO Message Stream
    5. Understanding Record Serialization
    6. KStream Methods
  7. Chapter 7 : Working with KStream
    1. Kafka Streams Exactly Once Implementation
    2. Implementing Exactly Once
    3. Let's Practice - a Complex Problem Statement
    4. Working with XML Inputs
    5. Handling Errors and Exceptions
    6. Mixed Branching of a KStream
    7. Handling Poisson Pills
  8. Chapter 8 : KTable and Aggregations
    1. Introducing KTable
    2. Deep Dive into KTable
    3. Computing Streaming Aggregates
    4. Aggregation Concepts
    5. Reducing a Kafka Stream
    6. Aggregating a Kafka Stream
    7. Aggregation Challenges
    8. KTable Aggregation
  9. Chapter 9 : Timestamp and Windowing Aggregates
    1. Kafka Time Semantics
    2. Windowing Aggregates
    3. Tumbling Window Versus Hopping Time Window
    4. Session Windows
  10. Chapter 10 : Joins in Kafka Streams
    1. Joins in Kafka Stream
    2. KStream to KStream Joins
    3. KTable to KTable Join
    4. KStream to KTable Join
    5. Implementing Complex Aggregation
  11. Chapter 11 : Kafka Streams in Functional Style and Unit Testing
    1. Stream Listener Manual Testing
    2. Stream Listeners Automating Test Cases
    3. Functional Style of Converting Stream Listeners
  12. Chapter 12 : Keep Learning
    1. Final Word

Product information

  • Title: Kafka Streams with Spring Cloud Stream
  • Author(s): Prashant Kumar Pandey
  • Release date: July 2021
  • Publisher(s): Packt Publishing
  • ISBN: 9781801811422