O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Apache Kafka Series - Kafka Streams for Data Processing

Apache Kafka Series - Kafka Streams for Data Processing

by Stéphane Maarek
Publisher: Packt Publishing
Release Date: May 2018
ISBN: 9781789343496
View table of contents
Start watching

Video Description

Learn the Kafka Streams API with hands-on examples. Learn exactly once, build and deploy apps with Java 8

About This Video

  • The easiest way to transform your data using the High Level DSL
  • Exactly once semantics support out of the box!
  • Deploy and scale your Kafka Streams application without a cluster!
  • Perform aggregations, joins, and any operations you may think of using only a few lines of code!
  • Built on top of Kafka, for fault tolerance, scalability and resiliency

In Detail

The new volume in the Apache Kafka Series! Learn the Kafka Streams data-processing library, for Apache Kafka. Join hundreds of knowledge savvy students in learning one of the most promising data-processing libraries on Apache Kafka. This course is based on Java 8, and will include one example in Scala. Kafka Streams is Java-based and therefore is not suited for any other programming language. This course is the first and only available Kafka Streams course on the web. Get it now to become a Kafka expert!

Downloading the example code for this course: You can download the example code files for all Packt video courses you have purchased from your account at http://www.PacktPub.com. If you purchased this course elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. Chapter 1 : Kafka Streams - First Look
    1. What is Kafka Streams? 00:04:47
    2. Course Objective / Prerequisites / Target Students 00:04:07
    3. About your Instructor: Stephane Maarek 00:00:50
    4. Running your first Kafka Streams Application: WordCount 00:12:53
    5. Kafka Streams vs other stream processing libraries (Spark Streaming, NiFi, Flink 00:02:56
    6. Code Download for this Class 00:00:55
  2. Chapter 2 : End to End Kafka Streams Application - Word Count
    1. Section Objective 00:01:46
    2. Kafka Streams Core Concepts 00:03:29
    3. Environment and IDE Setup: Java 8, Maven, IntelliJ IDEA 00:04:03
    4. Starter Project Setup 00:07:57
    5. Kafka Streams Application Properties 00:07:26
    6. Java 8 Lambda Functions - quick overview 00:02:56
    7. Word Count Application Topology 00:14:08
    8. Printing the Kafka Streams Topology 00:01:33
    9. Kafka Streams Graceful Shutdown 00:02:04
    10. Running Application from IntelliJ IDEA 00:09:22
    11. Debugging Application from IntelliJ IDEA 00:04:07
    12. Internal Topics for our Kafka Streams Application 00:03:48
    13. Packaging the application as Fat Jar & Running the Fat Jar 00:04:34
    14. Scaling our Application 00:04:00
    15. Section Wrap-Up 00:01:40
  3. Chapter 3 : KStreams and KTables Simple Operations (Stateless)
    1. Section Objectives 00:01:15
    2. KStream & KTables 00:03:39
    3. Stateless vs Stateful Operations 00:01:41
    4. MapValues / Map 00:01:36
    5. Filter / FilterNot 00:01:22
    6. FlatMapValues / FlatMap 00:02:10
    7. Branch 00:02:46
    8. SelectKey 00:01:15
    9. Reading from Kafka 00:02:16
    10. Writing to Kafka 00:01:52
    11. Streams Marked for Re-Partition 00:02:53
    12. Refresher on Log Compaction 00:17:45
    13. KStream and KTables Duality 00:02:21
    14. Transforming a KTable to a KStream 00:00:42
    15. Transforming a KStream to a KTable 00:01:24
    16. Section Summary 00:00:47
  4. Chapter 4 : Practice Exercise – FavouriteColour
    1. FavouriteColour - Practice Exercise Description & Guidance 00:03:20
    2. Stuck? Here are some Hints! 00:02:40
    3. Java Solution 00:08:34
    4. Running the application 00:05:46
    5. Scala Solution 00:06:21
  5. Chapter 5 : KStreams and KTables Advanced Operations (Stateful)
    1. Section Objective 00:01:00
    2. KTable groupBy 00:01:46
    3. KGroupedStream / KGroupedTable Count 00:02:05
    4. KGroupedStream / KGroupedTable Aggregate 00:04:13
    5. KGroupedStream / KGroupedTable Reduce 00:01:40
    6. KStream peek 00:02:08
    7. KStream Transform / TransformValues 00:01:13
    8. What if I want to write to an external System? 00:01:33
    9. Summary Diagram 00:01:04
  6. Chapter 6 : Exactly Once Semantics – Theory
    1. What's Exactly Once? 00:06:08
    2. Exactly Once in Kafka 0.11 00:02:42
    3. What's the problem with at least once anyway? 00:01:29
    4. How to do exactly once in Kafka Streams 00:02:05
  7. Chapter 7 : Exactly Once Semantics - Practice Exercise – BankBalance
    1. BankBalance - Exercise Overview 00:02:15
    2. Kafka Producer Guidance 00:01:42
    3. Kafka Producer Solution 00:14:18
    4. Kafka Streams Guidance & Hints 00:01:55
    5. Kafka Streams Solution 00:09:13
    6. Running the BankBalance Application 00:04:42
    7. Section Summary 00:00:57
  8. Chapter 8 : Joins - KStream to GlobalKTable example
    1. What are joins in Kafka Streams? 00:02:49
    2. Join Constraints and GlobalKTables 00:02:53
    3. The different types of joins: Inner Join, Left Join, Outer Join 00:02:46
    4. Creating a join with UserEnrich Kafka Streams App 00:12:33
    5. Running the Kafka Streams Join application 00:05:07
  9. Chapter 9 : Testing your Kafka Streams Application
    1. Testing in Kafka Streams 00:04:03
    2. Setup your Kafka Streams project 00:03:58
    3. Hands-On: Test your WordCount application 00:14:29
  10. Chapter 10 : Next Steps
    1. Congratulations and next steps 00:01:56