Video description
In this course, you will learn the basics of the Scala programming language; learn how Apache Spark operates on a cluster; set up discretized streams with Spark Streaming and transform them as data is received; analyze streaming data over sliding windows of time; maintain stateful information across streams of data; connect Spark Streaming with highly scalable sources of data, including Kafka, Flume, and Kinesis; dump streams of data in real-time to NoSQL databases such as Cassandra; run SQL queries on streamed data in real-time; train machine learning models in real-time with streaming data, and use them to make predictions that keep getting better over time; and also, package, deploy, and run self-contained Spark Streaming code to a real Hadoop cluster using Amazon Elastic MapReduce.
This course is very hands-on, filled with achievable activities and exercises to reinforce your learning. By the end of this course, you will be confidently creating Spark Streaming scripts in Scala and be prepared to tackle massive streams of data in a whole new way. You will be surprised at how easy Spark Streaming makes it!
What You Will Learn
- Process large amounts of real-time data using the Spark Streaming module
- Create efficient Spark applications using the Scala programming language
- Integrate Spark Streaming with various data sources
- Integrate Spark Streaming with Spark SQL to query your data in real time
- Train machine learning models with streaming data, and use for real-time predictions
- Maintain stateful data across a continuous stream of input data
Audience
If you are a student who wants to learn how to use Apache Spark or a big data professional who wants to process large amounts of data on a real-time basis, this course is for you. Some basic programming and scripting experience is required to get the most out of the course.
About The Author
Frank Kane: Frank Kane has spent nine years at Amazon and IMDb, developing and managing the technology that automatically delivers product and movie recommendations to hundreds of millions of customers all the time. He holds 17 issued patents in the fields of distributed computing, data mining, and machine learning. In 2012, Frank left to start his own successful company, Sundog Software, which focuses on virtual reality environment technology and teaches others about big data analysis.
Table of contents
- Chapter 1 : Getting Started
- Chapter 2 : A Crash Course in Scala
- Chapter 3 : Spark Streaming Concepts
- Chapter 4 : Spark Streaming Examples with Twitter
- Chapter 5 : Spark Streaming Examples with Clickstream / Apache Access Log Data
- Chapter 6 : Integrating with Other Systems
- Chapter 7 : Advanced Spark Streaming Examples
- Chapter 8 : Spark Streaming in Production
- Chapter 9 : You Made It!
Product information
- Title: Streaming Big Data with Spark Streaming, Scala, and Spark 3!
- Author(s):
- Release date: August 2022
- Publisher(s): Packt Publishing
- ISBN: 9781787123915
You might also like
video
Apache Spark with Scala - Learn Spark from a Big Data Guru
This course covers all the fundamentals of Apache Spark with Scala and teaches you everything you …
video
Scala & Spark-Master Big Data with Scala and Spark
The course Scala from Beginner to Pro is refreshingly different. The well-thought-out quizzes and mini projects …
video
Spark Programming in Scala for Beginners with Apache Spark 3
Apache Spark is a lightning-fast unified analytics engine for big data and machine learning. Since its …
video
50 Hours of Big Data, PySpark, AWS, Scala, and Scraping
Part 1 is designed to reflect the most in-demand Scala skills. It provides an in-depth understanding …