February 2022
Intermediate
4h 15m
English
on-demand course
Real-Time Stream Processing Using Apache Spark 3 for Scala Developers
with ScholarNest
Closed Captioning available in English
Overview
In this 3-hour course, you will learn how to create real-time streaming applications using Apache Spark 3. The course focuses on providing a deep understanding of Spark structured streaming APIs, handling streaming sources like Kafka, and mastering aggregate processing concepts, offering a complete skillset for real-time data processing.
What I will be able to do after this course
- Develop the ability to create streaming pipelines and handle various data streams efficiently.
- Gain proficiency in integrating Apache Kafka with Apache Spark for optimized stream processing.
- Learn techniques for performing stateful and stateless transformations and processing pipelines.
- Master strategies to handle scalability and memory challenges in streaming applications.
- Understand advanced streaming concepts like watermarking and aggregate windowing for real-time analytics.
Course Instructor(s)
Your instructor for this course is an experienced software engineer and educator, specializing in big data technologies and distributed systems. With practical experience in implementing systems using Apache Spark and Kafka, they bring real-world expertise to the instructional content. Their educational approach involves hands-on demonstrations and breakdowns of complex technical topics into understandable segments, ensuring learners acquire practical, applicable skills.
Who is it for?
This course is designed for software engineers, data engineers, and programmers who wish to develop real-time data processing applications. It is ideal for those with foundational knowledge in Scala and some experience in distributed computing. Learners aiming to delve into big data engineering or enhance their skills in technologies like Apache Spark will find this course particularly beneficial. Additionally, architects or managers working with distributed system implementations can also benefit.
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Watch now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.