Learning Apache Storm for Big Data Processing

Learning Apache Storm for Big Data Processing

by Prashant Nair
Released July 2018
Publisher(s): Packt Publishing
ISBN: 9781786460028

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to live online training experiences, plus books, videos, and digital content from O’Reilly and nearly 200 trusted publishing partners.

Start your free trial

Video description

Using a free and open source, distributed real-time computation system was never this much fun!

About This Video

  • Learn the key points of the Lambda Architecture and the role Apache Storm plays in it
  • Learn programming techniques to create a Storm topology, its patterns, and anti-patterns.
  • Understanding the core components and Apache Storm daemons
  • Achieving fault-tolerance in Apache Storm

In Detail

Apache Storm is a distributed real-time processing engine. Created by Nathanmarz for Backtype and later open sourced under Apache License 2, it's a scalable and a fault-tolerant engine used to process a massive number of unbounded streams. In this course you will see how simple yet efficient Apache Storm is when it comes to real-time processing.

In the course, you will learn about data processing types followed by Apache Storm and its features. You'll learn the core concepts of Apache Storm such as spouts, bolts, topology, and stream grouping, and set up Apache Storm in single-node and multi-node configurations. Also you'll explore how fault-tolerant Apache Storm is.

Taking this course will kick-start your experience with Apache Storm; you'll create a scalable, fault-tolerant, real-time processing application while setting a strong base for the fundamentals of the real-time processing paradigm and Apache Storm.

Audience

This course targets data engineers, solution architects, Java developers, technical managers, and programming analysts working in a big data setup or anyone who is keen to learn the ins and outs of big data processing.

The audience is expected to have a working knowledge of programming with Java. A basic knowledge of Apache Maven (and a working knowledge of Eclipse and Linux terminals) will help you understand its concepts a lot more easily.

Publisher resources

Download Example Code

Table of contents

  1. Chapter 1 : Introducing Real-time Processing
    1. The Course Overview
    2. Understanding Lambda Architecture
    3. Big Data Processing Types
    4. What Is Apache Storm?
    5. When to Use Apache Storm?
  2. Chapter 2 : Apache Storm Concepts
    1. Topology
    2. Tuples
    3. Spouts and Bolts
    4. Streams and StreamGrouping
  3. Chapter 3 : Setting Up Your Apache Storm Development Environment
    1. Introduction – Prerequisites and System Requirements
    2. Installing Java and Setting Environment Variables
    3. Installing and Configuring Eclipse
    4. Building Apache Storm Project Using Maven
    5. Building Apache Storm Project Using External JAR Configuration
  4. Chapter 4 : Creating Our First Storm Topology
    1. Understanding the Problem Statement
    2. Developing Spout Class to Emit the Data
    3. Develop a Bolt Class to Perform Calculation
    4. Develop a Bolt Class to Print Result in Console
    5. Developing Topology Class
    6. Executing Our Application in Eclipse
  5. Chapter 5 : Setting Up Apache Storm as a Single-Node Cluster
    1. Understanding Storm Daemons
    2. Prerequisites
    3. Setting Up Zookeeper in Standalone Mode
    4. Install and Configure Apache Storm in Single-Node
    5. Deploy NumSquareTopology in Cluster
    6. Explore Storm UI and Understand Essential Features
  6. Chapter 6 : Setting Up Apache Storm in Multi-Node Cluster
    1. Setting Up Zookeeper in Multi-Node Mode
    2. Setting Up Apache Storm in Multi-Node Cluster
  7. Chapter 7 : Implementing StreamGroupings in Apache Storm
    1. Introduction
    2. Implementing ShuffleGrouping
    3. Implementing FieldGrouping
    4. Implementing AllGrouping
    5. Implementing CustomGrouping
    6. Implementing DirectGrouping
  8. Chapter 8 : Integrating Hadoop with Apache Storm
    1. Introduction
    2. Writing a HDFS Bolt
  9. Chapter 9 : Understanding and Implementing Tridents in Apache Storm
    1. Introduction
    2. Building Topology Using Trident
    3. Understand and Implement Map, Filter, and Aggregate Function
    4. Windowing Operations
    5. Joining Stream Tuples in Storm

Product information

  • Title: Learning Apache Storm for Big Data Processing
  • Author(s): Prashant Nair
  • Release date: July 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781786460028