Apache Kafka Series - Kafka Streams for Data Processing

Apache Kafka Series - Kafka Streams for Data Processing

by Stéphane Maarek
Released May 2018
Publisher(s): Packt Publishing
ISBN: 9781789343496

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to live online training experiences, plus books, videos, and digital content from O’Reilly and nearly 200 trusted publishing partners.

Start your free trial

Video description

Learn the Kafka Streams API with hands-on examples. Learn exactly once, build and deploy apps with Java 8

About This Video

  • The easiest way to transform your data using the High Level DSL
  • Exactly once semantics support out of the box!
  • Deploy and scale your Kafka Streams application without a cluster!
  • Perform aggregations, joins, and any operations you may think of using only a few lines of code!
  • Built on top of Kafka, for fault tolerance, scalability and resiliency

In Detail

The new volume in the Apache Kafka Series! Learn the Kafka Streams data-processing library, for Apache Kafka. Join hundreds of knowledge savvy students in learning one of the most promising data-processing libraries on Apache Kafka. This course is based on Java 8, and will include one example in Scala. Kafka Streams is Java-based and therefore is not suited for any other programming language. This course is the first and only available Kafka Streams course on the web. Get it now to become a Kafka expert!

Who this book is for

Developers and devops who would like to learn how to write, package, deploy and run a Kafka Streams applications. Architects who would like to understand how Kafka Streams works and its position in the Kafka-centered data pipeline and enterprise architecture. People who fully understand the basics of Kafka. Java 8 and Scala developers only

Publisher resources

Download Example Code

Table of contents

  1. Chapter 1 : Kafka Streams - First Look
    1. What is Kafka Streams?
    2. Course Objective / Prerequisites / Target Students
    3. About your Instructor: Stephane Maarek
    4. Running your first Kafka Streams Application: WordCount
    5. Kafka Streams vs other stream processing libraries (Spark Streaming, NiFi, Flink
    6. Code Download for this Class
  2. Chapter 2 : End to End Kafka Streams Application - Word Count
    1. Section Objective
    2. Kafka Streams Core Concepts
    3. Environment and IDE Setup: Java 8, Maven, IntelliJ IDEA
    4. Starter Project Setup
    5. Kafka Streams Application Properties
    6. Java 8 Lambda Functions - quick overview
    7. Word Count Application Topology
    8. Printing the Kafka Streams Topology
    9. Kafka Streams Graceful Shutdown
    10. Running Application from IntelliJ IDEA
    11. Debugging Application from IntelliJ IDEA
    12. Internal Topics for our Kafka Streams Application
    13. Packaging the application as Fat Jar Running the Fat Jar
    14. Scaling our Application
    15. Section Wrap-Up
  3. Chapter 3 : KStreams and KTables Simple Operations (Stateless)
    1. Section Objectives
    2. KStream KTables
    3. Stateless vs Stateful Operations
    4. MapValues / Map
    5. Filter / FilterNot
    6. FlatMapValues / FlatMap
    7. Branch
    8. SelectKey
    9. Reading from Kafka
    10. Writing to Kafka
    11. Streams Marked for Re-Partition
    12. Refresher on Log Compaction
    13. KStream and KTables Duality
    14. Transforming a KTable to a KStream
    15. Transforming a KStream to a KTable
    16. Section Summary
  4. Chapter 4 : Practice Exercise – FavouriteColour
    1. FavouriteColour - Practice Exercise Description Guidance
    2. Stuck? Here are some Hints!
    3. Java Solution
    4. Running the application
    5. Scala Solution
  5. Chapter 5 : KStreams and KTables Advanced Operations (Stateful)
    1. Section Objective
    2. KTable groupBy
    3. KGroupedStream / KGroupedTable Count
    4. KGroupedStream / KGroupedTable Aggregate
    5. KGroupedStream / KGroupedTable Reduce
    6. KStream peek
    7. KStream Transform / TransformValues
    8. What if I want to write to an external System?
    9. Summary Diagram
  6. Chapter 6 : Exactly Once Semantics – Theory
    1. What's Exactly Once?
    2. Exactly Once in Kafka 0.11
    3. What's the problem with at least once anyway?
    4. How to do exactly once in Kafka Streams
  7. Chapter 7 : Exactly Once Semantics - Practice Exercise – BankBalance
    1. BankBalance - Exercise Overview
    2. Kafka Producer Guidance
    3. Kafka Producer Solution
    4. Kafka Streams Guidance Hints
    5. Kafka Streams Solution
    6. Running the BankBalance Application
    7. Section Summary
  8. Chapter 8 : Joins - KStream to GlobalKTable example
    1. What are joins in Kafka Streams?
    2. Join Constraints and GlobalKTables
    3. The different types of joins: Inner Join, Left Join, Outer Join
    4. Creating a join with UserEnrich Kafka Streams App
    5. Running the Kafka Streams Join application
  9. Chapter 9 : Testing your Kafka Streams Application
    1. Testing in Kafka Streams
    2. Setup your Kafka Streams project
    3. Hands-On: Test your WordCount application
  10. Chapter 10 : Next Steps
    1. Congratulations and next steps

Product information

  • Title: Apache Kafka Series - Kafka Streams for Data Processing
  • Author(s): Stéphane Maarek
  • Release date: May 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789343496