Distributed Systems with Node.js

Distributed Systems with Node.js

by Thomas Hunter
Released November 2020
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781492077244

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Many companies, from startups to Fortune 500 companies alike, use Node.js to build performant backend services. And engineers love Node.js for its approachable API and familiar syntax. Backed by the world's largest package repository, Node's enterprise foothold is only expected to grow.

In this hands-on guide, author Thomas Hunter II proves that Node.js is just as capable as traditional enterprise platforms for building services that are observable, scalable, and resilient. Intermediate to advanced Node.js developers will find themselves integrating application code with a breadth of tooling from each layer of a modern service stack.

  • Learn why running redundant copies of the same Node.js service is necessary
  • Know which protocol to choose, depending on the situation
  • Fine-tune your application containers for use in production
  • Track down errors in a distributed setting to determine which service is at fault
  • Simplify app code and increase performance by offloading work to a reverse proxy
  • Build dashboards to monitor service health and throughput
  • Find out why so many different tools are required when operating in an enterprise environment

Table of contents

  1. Foreword
  2. Preface
    1. Target Audience
    2. Goals
    3. Conventions Used in This Book
    4. Using Code Examples
    5. O’Reilly Online Learning
    6. How to Contact Us
    7. Acknowledgments
  3. 1. Why Distributed?
    1. The Single-Threaded Nature of JavaScript
    2. Quick Node.js Overview
    3. The Node.js Event Loop
      1. Event Loop Phases
      2. Code Example
      3. Event Loop Tips
    4. Sample Applications
      1. Service Relationship
      2. Producer Service
      3. Consumer Service
  4. 2. Protocols
    1. Request and Response with HTTP
      1. HTTP Payloads
      2. HTTP Semantics
      3. HTTP Compression
      4. HTTPS / TLS
      5. JSON over HTTP
      6. The Dangers of Serializing POJOs
    2. API Facade with GraphQL
      1. GraphQL Schema
      2. Queries and Responses
      3. GraphQL Producer
      4. GraphQL Consumer
    3. RPC with gRPC
      1. Protocol Buffers
      2. gRPC Producer
      3. gRPC Consumer
  5. 3. Scaling
    1. The Cluster Module
      1. A Simple Example
      2. Request Dispatching
      3. Cluster Shortcomings
    2. Reverse Proxies with HAProxy
      1. Introduction to HAProxy
      2. Load Balancing and Health Checks
      3. Compression
      4. TLS Termination
      5. Rate Limiting and Back Pressure
    3. SLA and Load Testing
      1. Introduction to Autocannon
      2. Running a Baseline Load Test
      3. Reverse Proxy Concerns
      4. Protocol Concerns
      5. Coming Up with SLOs
  6. 4. Observability
    1. Environments
    2. Logging with ELK
      1. Running ELK via Docker
      2. Transmitting Logs from Node.js
      3. Creating a Kibana Dashboard
      4. Running Ad-Hoc Queries
    3. Metrics with Graphite, StatsD, and Grafana
      1. Running via Docker
      2. Transmitting Metrics from Node.js
      3. Creating a Grafana Dashboard
      4. Node.js Health Indicators
    4. Distributed Request Tracing with Zipkin
      1. How Does Zipkin Work?
      2. Running Zipkin via Docker
      3. Transmitting Traces from Node.js
      4. Visualizing a Request Tree
      5. Visualizing Microservice Dependencies
    5. Health Checks
      1. Building a Health Check
      2. Testing the Health Check
    6. Alerting with Cabot
      1. Create a Twilio Trial Account
      2. Running Cabot via Docker
      3. Creating a Health Check
  7. 5. Containers
    1. Introduction to Docker
    2. Containerizing a Node.js Service
      1. Dependency Stage
      2. Release Stage
      3. From Image to Container
      4. Rebuilding and Versioning an Image
    3. Basic Orchestration with Docker Compose
      1. Composing Node.js Services
    4. Internal Docker Registry
      1. Running the Docker Registry
      2. Pushing and Pulling to the Registry
      3. Running a Docker Registry UI
  8. 6. Deployments
    1. Build Pipeline with Travis CI
      1. Creating a Basic Project
      2. Configuring Travis CI
      3. Testing a Pull Request
    2. Automated Testing
      1. Unit Tests
      2. Integration Tests
      3. Code Coverage Enforcement
    3. Deploying to Heroku
      1. Create a Heroku App
      2. Configure Travis CI
      3. Deploy Your Application
    4. Modules, Packages, and SemVer
      1. Node.js Modules
      2. SemVer (Semantic Versioning)
      3. npm Packages and the npm CLI
    5. Internal npm Registry
      1. Running Verdaccio
      2. Configuring npm to Use Verdaccio
      3. Publishing to Verdaccio
  9. 7. Container Orchestration
    1. Introduction to Kubernetes
      1. Kubernetes Overview
      2. Kubernetes Concepts
      3. Starting Kubernetes
    2. Getting Started
    3. Deploying an Application
      1. Kubectl Subcommands
      2. Kubectl Configuration Files
    4. Service Discovery
    5. Modifying Deployments
      1. Scaling Application Instances
      2. Deploying New Application Versions
      3. Rolling Back Application Deployments
  10. 8. Resilience
    1. The Death of a Node.js Process
      1. Process Exit
      2. Exceptions, Rejections, and Emitted Errors
      3. Signals
    2. Building Stateless Services
      1. Avoiding Memory Leaks
      2. Bounded In-Process Caches
    3. External Caching with Memcached
      1. Introducing Memcached
      2. Running Memcached
      3. Caching Data with Memcached
      4. Data Structure Mutations
    4. Database Connection Resilience
      1. Running PostgreSQL
      2. Automatic Reconnection
      3. Connection Pooling
    5. Schema Migrations with Knex
      1. Configuring Knex
      2. Creating a Schema Migration
      3. Applying a Migration
      4. Rolling Back a Migration
      5. Live Migrations
    6. Idempotency and Messaging Resilience
      1. HTTP Retry Logic
      2. Circuit Breaker Pattern
      3. Exponential Backoff
    7. Resilience Testing
      1. Random Crashes
      2. Event Loop Pauses
      3. Random Failed Async Operations
  11. 9. Distributed Primitives
    1. The ID Generation Problem
    2. Introduction to Redis
    3. Redis Operations
      1. Strings
      2. Lists
      3. Sets
      4. Hash
      5. Sorted Sets
      6. Generic Commands
      7. Other Types
    4. Seeking Atomicity
    5. Transactions
    6. Lua Scripting
      1. Writing a Lua Script File
      2. Loading the Lua Script
      3. Tying It All Together
  12. 10. Security
    1. Wrangling Repositories
    2. Recognizing Attack Surface
      1. Parameter Checking and Deserialization
      2. Malicious npm Packages
    3. Application Configuration
      1. Environment Variables
      2. Configuration Files
      3. Secrets Management
    4. Upgrading Dependencies
      1. Automatic Upgrades with GitHub Dependabot
      2. Manual Upgrades with npm CLI
      3. Unpatched Vulnerabilities
    5. Upgrading Node.js
      1. Node.js LTS Schedule
      2. Upgrade Approach
  13. A. Installing HAProxy
    1. Linux: Build from Source
    2. Linux: Install Precompiled Binary
    3. macOS: Install via Homebrew
  14. B. Installing Docker
    1. macOS: Install Docker Desktop for Mac
    2. Linux: Convenient Install Script
  15. C. Installing Minikube & Kubectl
    1. Linux: Debian Package and Precompiled Binary
    2. macOS: Install via Homebrew
  16. Index

Product information

  • Title: Distributed Systems with Node.js
  • Author(s): Thomas Hunter
  • Release date: November 2020
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492077244