Frequently, Kafka is just a piece of the stack that lives in production that often times no one wants to touch—because it just works. Kafka sits at the core of AppsFlyer’s infrastructure that processes billions of events daily.
Alon Gavra (AppsFlyer) dives into how AppsFlyer built its microservices architecture with Kafka as its core piece to support 70B+ requests daily. With continuous growth, the company needed to “learn on the job” how to improve its Kafka architecture by moving to the producer-owner cluster model, breaking up its massive monolith clusters to smaller, more robust clusters and migrating from an older version of Kafka with real-time production clients and data streams. Alon outlines best practices for leveraging Kafka’s in-memory capabilities and built-in partitioning, as well as some of the tweaks and stabilization mechanisms that enable real-time performance at web scale, alongside processes for continuous upgrades and deployments with end-to-end automation, in an environment of constant traffic growth.
- A basic understanding of Kafka
What you'll learn
- Learn why Kafka is useful for handling traffic spikes and ensuring no data loss in constant growth
This session is from the 2019 O'Reilly Strata Conference in New York, NY.
- Title: Managing your Kafka in an explosive growth environment
- Release date: February 2020
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 0636920372073
You might also like
Software Architecture Fundamentals, Second Edition
Being a successful software architect is more than just possessing technical knowledge. It’s about thinking like …
Microservices designs require you to change how you build and deploy applications. Instead of creating a …
Software Engineering at Google
Today, software engineers need to know not only how to program effectively but also how to …
Amazon Web Services AWS LiveLessons 2nd Edition
More Than 17 Hours of Video Instruction More than 17 hours of video instruction on Amazon …