O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Learning Apache Storm for Big Data Processing

Learning Apache Storm for Big Data Processing

by Prashant Nair
Publisher: Packt Publishing
Release Date: July 2018
ISBN: 9781786460028
View table of contents
Start watching

Video Description

Using a free and open source, distributed real-time computation system was never this much fun!

About This Video

  • Learn the key points of the Lambda Architecture and the role Apache Storm plays in it
  • Learn programming techniques to create a Storm topology, its patterns, and anti-patterns.
  • Understanding the core components and Apache Storm daemons
  • Achieving fault-tolerance in Apache Storm

In Detail

Apache Storm is a distributed real-time processing engine. Created by Nathanmarz for Backtype and later open sourced under Apache License 2, it's a scalable and a fault-tolerant engine used to process a massive number of unbounded streams. In this course you will see how simple yet efficient Apache Storm is when it comes to real-time processing.

In the course, you will learn about data processing types followed by Apache Storm and its features. You'll learn the core concepts of Apache Storm such as spouts, bolts, topology, and stream grouping, and set up Apache Storm in single-node and multi-node configurations. Also you'll explore how fault-tolerant Apache Storm is.

Taking this course will kick-start your experience with Apache Storm; you'll create a scalable, fault-tolerant, real-time processing application while setting a strong base for the fundamentals of the real-time processing paradigm and Apache Storm.

All the code and supporting files for the course can be found here- https://github.com/PacktPublishing/Learning-Apache-Storm-for-Big-Data-Processing

Table of Contents

  1. Chapter 1 : Introducing Real-time Processing
    1. The Course Overview 00:03:54
    2. Understanding Lambda Architecture 00:02:16
    3. Big Data Processing Types 00:02:06
    4. What Is Apache Storm? 00:00:48
    5. When to Use Apache Storm? 00:01:20
  2. Chapter 2 : Apache Storm Concepts
    1. Topology 00:03:25
    2. Tuples 00:02:00
    3. Spouts and Bolts 00:04:50
    4. Streams and StreamGrouping 00:01:43
  3. Chapter 3 : Setting Up Your Apache Storm Development Environment
    1. Introduction – Prerequisites and System Requirements 00:02:32
    2. Installing Java and Setting Environment Variables 00:04:03
    3. Installing and Configuring Eclipse 00:03:16
    4. Building Apache Storm Project Using Maven 00:04:19
    5. Building Apache Storm Project Using External JAR Configuration 00:03:41
  4. Chapter 4 : Creating Our First Storm Topology
    1. Understanding the Problem Statement 00:03:25
    2. Developing Spout Class to Emit the Data 00:06:16
    3. Develop a Bolt Class to Perform Calculation 00:03:38
    4. Develop a Bolt Class to Print Result in Console 00:02:33
    5. Developing Topology Class 00:04:01
    6. Executing Our Application in Eclipse 00:03:01
  5. Chapter 5 : Setting Up Apache Storm as a Single-Node Cluster
    1. Understanding Storm Daemons 00:03:56
    2. Prerequisites 00:01:52
    3. Setting Up Zookeeper in Standalone Mode 00:04:55
    4. Install and Configure Apache Storm in Single-Node 00:09:01
    5. Deploy NumSquareTopology in Cluster 00:04:02
    6. Explore Storm UI and Understand Essential Features 00:03:08
  6. Chapter 6 : Setting Up Apache Storm in Multi-Node Cluster
    1. Setting Up Zookeeper in Multi-Node Mode 00:08:02
    2. Setting Up Apache Storm in Multi-Node Cluster 00:04:34
  7. Chapter 7 : Implementing StreamGroupings in Apache Storm
    1. Introduction 00:02:07
    2. Implementing ShuffleGrouping 00:05:33
    3. Implementing FieldGrouping 00:03:47
    4. Implementing AllGrouping 00:01:58
    5. Implementing CustomGrouping 00:03:35
    6. Implementing DirectGrouping 00:03:36
  8. Chapter 8 : Integrating Hadoop with Apache Storm
    1. Introduction 00:09:42
    2. Writing a HDFS Bolt 00:06:39
  9. Chapter 9 : Understanding and Implementing Tridents in Apache Storm
    1. Introduction 00:01:51
    2. Building Topology Using Trident 00:02:55
    3. Understand and Implement Map, Filter, and Aggregate Function 00:05:26
    4. Windowing Operations 00:03:13
    5. Joining Stream Tuples in Storm 00:02:56