Apache Kafka Series - Kafka Streams for Data Processing

Apache Kafka Series - Kafka Streams for Data Processing

by Stéphane Maarek
Released May 2018
Publisher(s): Packt Publishing
ISBN: 9781789343496

Explore a preview version of Apache Kafka Series - Kafka Streams for Data Processing right now.

O’Reilly members get unlimited access to live online training experiences, plus books, videos, and digital content from 200+ publishers.

Start your free trial

Video description

Learn the Kafka Streams API with hands-on examples. Learn exactly once, build and deploy apps with Java 8

About This Video

  • The easiest way to transform your data using the High Level DSL
  • Exactly once semantics support out of the box!
  • Deploy and scale your Kafka Streams application without a cluster!
  • Perform aggregations, joins, and any operations you may think of using only a few lines of code!
  • Built on top of Kafka, for fault tolerance, scalability and resiliency

In Detail

The new volume in the Apache Kafka Series! Learn the Kafka Streams data-processing library, for Apache Kafka. Join hundreds of knowledge savvy students in learning one of the most promising data-processing libraries on Apache Kafka. This course is based on Java 8, and will include one example in Scala. Kafka Streams is Java-based and therefore is not suited for any other programming language. This course is the first and only available Kafka Streams course on the web. Get it now to become a Kafka expert!

Downloading the example code for this course: You can download the example code files for all Packt video courses you have purchased from your account at http://www.PacktPub.com. If you purchased this course elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Publisher resources

Download Example Code

Table of contents

  1. Chapter 1 : Kafka Streams - First Look
    1. What is Kafka Streams? 00:04:47
    2. Course Objective / Prerequisites / Target Students 00:04:07
    3. About your Instructor: Stephane Maarek 00:00:50
    4. Running your first Kafka Streams Application: WordCount 00:12:53
    5. Kafka Streams vs other stream processing libraries (Spark Streaming, NiFi, Flink 00:02:56
    6. Code Download for this Class 00:00:55
  2. Chapter 2 : End to End Kafka Streams Application - Word Count
    1. Section Objective 00:01:46
    2. Kafka Streams Core Concepts 00:03:29
    3. Environment and IDE Setup: Java 8, Maven, IntelliJ IDEA 00:04:03
    4. Starter Project Setup 00:07:57
    5. Kafka Streams Application Properties 00:07:26
    6. Java 8 Lambda Functions - quick overview 00:02:56
    7. Word Count Application Topology 00:14:08
    8. Printing the Kafka Streams Topology 00:01:33
    9. Kafka Streams Graceful Shutdown 00:02:04
    10. Running Application from IntelliJ IDEA 00:09:22
    11. Debugging Application from IntelliJ IDEA 00:04:07
    12. Internal Topics for our Kafka Streams Application 00:03:48
    13. Packaging the application as Fat Jar & Running the Fat Jar 00:04:34
    14. Scaling our Application 00:04:00
    15. Section Wrap-Up 00:01:40
  3. Chapter 3 : KStreams and KTables Simple Operations (Stateless)
    1. Section Objectives 00:01:15
    2. KStream & KTables 00:03:39
    3. Stateless vs Stateful Operations 00:01:41
    4. MapValues / Map 00:01:36
    5. Filter / FilterNot 00:01:22
    6. FlatMapValues / FlatMap 00:02:10
    7. Branch 00:02:46
    8. SelectKey 00:01:15
    9. Reading from Kafka 00:02:16
    10. Writing to Kafka 00:01:52
    11. Streams Marked for Re-Partition 00:02:53
    12. Refresher on Log Compaction 00:17:45
    13. KStream and KTables Duality 00:02:21
    14. Transforming a KTable to a KStream 00:00:42
    15. Transforming a KStream to a KTable 00:01:24
    16. Section Summary 00:00:47
  4. Chapter 4 : Practice Exercise – FavouriteColour
    1. FavouriteColour - Practice Exercise Description & Guidance 00:03:20
    2. Stuck? Here are some Hints! 00:02:40
    3. Java Solution 00:08:34
    4. Running the application 00:05:46
    5. Scala Solution 00:06:21
  5. Chapter 5 : KStreams and KTables Advanced Operations (Stateful)
    1. Section Objective 00:01:00
    2. KTable groupBy 00:01:46
    3. KGroupedStream / KGroupedTable Count 00:02:05
    4. KGroupedStream / KGroupedTable Aggregate 00:04:13
    5. KGroupedStream / KGroupedTable Reduce 00:01:40
    6. KStream peek 00:02:08
    7. KStream Transform / TransformValues 00:01:13
    8. What if I want to write to an external System? 00:01:33
    9. Summary Diagram 00:01:04
  6. Chapter 6 : Exactly Once Semantics – Theory
    1. What's Exactly Once? 00:06:08
    2. Exactly Once in Kafka 0.11 00:02:42
    3. What's the problem with at least once anyway? 00:01:29
    4. How to do exactly once in Kafka Streams 00:02:05
  7. Chapter 7 : Exactly Once Semantics - Practice Exercise – BankBalance
    1. BankBalance - Exercise Overview 00:02:15
    2. Kafka Producer Guidance 00:01:42
    3. Kafka Producer Solution 00:14:18
    4. Kafka Streams Guidance & Hints 00:01:55
    5. Kafka Streams Solution 00:09:13
    6. Running the BankBalance Application 00:04:42
    7. Section Summary 00:00:57
  8. Chapter 8 : Joins - KStream to GlobalKTable example
    1. What are joins in Kafka Streams? 00:02:49
    2. Join Constraints and GlobalKTables 00:02:53
    3. The different types of joins: Inner Join, Left Join, Outer Join 00:02:46
    4. Creating a join with UserEnrich Kafka Streams App 00:12:33
    5. Running the Kafka Streams Join application 00:05:07
  9. Chapter 9 : Testing your Kafka Streams Application
    1. Testing in Kafka Streams 00:04:03
    2. Setup your Kafka Streams project 00:03:58
    3. Hands-On: Test your WordCount application 00:14:29
  10. Chapter 10 : Next Steps
    1. Congratulations and next steps 00:01:56

Product information

  • Title: Apache Kafka Series - Kafka Streams for Data Processing
  • Author(s): Stéphane Maarek
  • Release date: May 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789343496