Video description
This is the new volume in the Apache Kafka Series! Learn Apache Avro, the confluent schema registry for Apache Kafka and the confluent REST proxy for Apache Kafka. Join hundreds of knowledge savvy students in learning some of the most important components in a typical Apache Kafka stack. Apache Avro is one of the most powerful and most popular fast data serialisation mechanism with Apache Kafka: Schema is a first class citizen of the data; data serialisation format that is lightweight; documentation is embedded in the schema; easy to read and write using many languages; fast and big data de-facto data serialisation format. Confluent schema registry for Apache Kafka is the de-facto standard way of storing Avro schemas for your Apache Kafka topics: Stores a versioned history of all your schemas in Apache Kafka; supports and enforces conditions for schema evolution (backward, forward, full compatibility): Kafka Avro serialiser and deserialiser automatically integrate with Apache Kafka and confluent schema registry; make your messages much smaller and lighter! Perfect for increasing Apache Kafka throughput and performance. Confluent REST Proxy is the perfect way to communicate for sending Avro data using non Java languages to Apache Kafka: Write and read binary, JSON and Avro data to Apache Kafka using an HTTP REST API; interact with Apache Kafka using any programming language (not just Java); consult topic list and topic metadata in Apache Kafka
What You Will Learn
- Write simple and complex Avro schemas
- Create, write and read Avro objects in Java
- Write a Java producer and consumer leveraging Avro data and the schema registry
- Learn about schema evolution
- Perform schema evolution using the command line and in Java
- Utilize the REST proxy using a REST client
Audience
Developers who would like to learn how to write and run an application that leverages Avro and the schema registry. Architects who want to understand the role of the schema registry and REST proxy in an enterprise pipeline. This course is intended for people who fully understand the basics of Kafka. People who want to learn the confluent components: schema registry and REST proxy
About The Author
Stéphane Maarek: Stéphane Maarek is a solutions architect, consultant, and software developer who has a particular interest in all things related to big data and analytics. He is also a bestseller instructor on Udemy for his courses on Apache Kafka, Apache NiFi, and AWS Lambda. He loves Apache Kafka and regularly contributes to the Apache Kafka project.
Stéphane has also written a guest blog post that was featured on the Confluent website, the company behind Apache Kafka. He is also an AWS Certified Solutions Architect and has many years of experience with technologies such as Apache Kafka, Apache NiFi, Apache Spark, Hadoop, PostgreSQL, Tableau, Spotfire, Docker, Ansible, and more.
Table of contents
- Chapter 1 : Course Introduction
- Chapter 2 : Avro Schemas
- Chapter 3 : Avro in Java
- Chapter 4 : Setup and Launch Kafka
-
Chapter 5 : Confluent Schema Registry and Kafka
- Confluent Schema Registry
- Kafka Avro Console Producer Consumer
- Writing a Kafka Avro Producer in Java
- Writing a Kafka Avro Consumer in Java
- Reminder on Schema Evolution
- Writing a V2 Kafka Producer
- Writing a V2 Kafka Consumer
- Summary on Compatibility Changes
- Kafka Schema Registry Deep Dive
- Managing Schemas Efficiently Section Summary
-
Chapter 6 : Confluent REST Proxy
- Kafka REST Proxy Introduction and Purpose
- V1 vs V2 APIs
- Insomnia Setup (REST Client)
- Topic Operations
- Producing in Binary with the Kafka REST Proxy
- Consuming in Binary with the Kafka REST Proxy
- Producing in JSON with the Kafka REST Proxy
- Consuming in JSON with the Kafka REST Proxy
- Producing in Avro with the Kafka REST Proxy
- Consuming in Avro with the Kafka REST Proxy
- Chapter 7 : Annexes
Product information
- Title: Apache Kafka Series - Confluent Schema Registry and REST Proxy
- Author(s):
- Release date: May 2018
- Publisher(s): Packt Publishing
- ISBN: 9781789344806
You might also like
video
Apache Kafka Series - Kafka Cluster Setup and Administration
This course is designed for system administrators, architects and developers. It includes a step-by-step guide for …
video
Apache Kafka Fundamentals LiveLessons
Almost 4 hours of video instruction to help you learn the key concepts and features behind …
video
Apache Kafka - Real-time Stream Processing (Master Class)
If you want to understand the concept of stream processing, this course is for you. Using …
video
Apache Kafka Series - Kafka Monitoring and Operations
This course helps you learn Kafka administration, Kafka monitoring, Kafka operations, and Kafka upgrades. It covers …