O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Linux Clustering: Building and Maintaining Linux Clusters

Book Description

Linux Clustering: Building and Maintaining Linux Clusters provides Linux users with information about building their own Linux cluster from the ground up. It gives best practices, helpful hints, and guidelines about building one server or hundreds of servers at a level that administrators at any experience level can understand.

From installation of the air conditioning and power in the data center, to alternative file systems to the final production run, this book provides you with everything that you need to know. Linux Clustering: Building and Maintaining Linux Clusters walks you through the initial design and selection of the best possible types of clusters, as well as covering monitoring tools and providing for disaster recovery. Not only does this book provide information on parallel and Beowulf type clusters, Charles Bookman goes into depth on high availability clusters, load balancing, and provides advice for writing your own distributing applications as well. Incorporating best practices and cutting-edge approaches, Bookman provides step-by-step and tried-and-true methods of bringing up a Linux cluster to production level.

Table of Contents

  1. Copyright
    1. Dedication
  2. About the Author
  3. About the Technical Reviewers
  4. Acknowledgments
  5. Tell Us What You Think
  6. Introduction
    1. Who Should Read This Book
    2. Who This Book Is Not For
    3. Overview
    4. Conventions
  7. I. An Overview of Clustering for Linux, Preparing Your Network, and Linux Servers
    1. 1. Clustering Fundamentals
      1. Clustering for the Enterprise
        1. How Clustering Can Help
        2. Using Linux for Clustering
          1. Disadvantages of Using Linux
      2. Clusters Aren’t Just for High Performance Anymore
        1. High Availability and Fault-Tolerant Clusters
          1. Single Point of Failure
          2. Server Redundancy
          3. Managing Shared Storage and Dynamic Data
        2. Load Balancing
        3. Distributed Computing
        4. Parallel Computing
          1. How Now, Brown Cow
          2. Amdahl’s Law
      3. SMP and the Primary Processor Paradigm
      4. Quick and Dirty Clustering
        1. Dial Ping for Service
        2. Load Balancing Using DNS Records
      5. Alternate File Systems
        1. Journaling File Systems
        2. Network File Systems
      6. Networking 101 for Clusters
        1. The OSI Networking Model
        2. Network Topology
        3. Services to Consider
        4. Keeping Your Services Off the Public Network
      7. Summary
    2. 2. Preparing Your Linux Cluster
      1. Planning the Topology
        1. One Format Yields One Solution
        2. Get the Best Components for the Price
          1. Replacement Considerations
          2. Avoid the Single Point of Failure
        3. Plan for Administrative Overhead
      2. Select the Right Distribution
        1. Differences in Distributions
        2. Distributions to Avoid
      3. Planning the Environment
        1. Consider Power and Cooling Requirements
          1. Electricity Requirements
          2. Provisioning for Cooling
          3. Space: The Final Frontier
        2. Racks for Servers and Desktops
          1. Switches and Connectivity
      4. A Few Words About Security
        1. Security in Layers
          1. Security at the System Level
          2. Security at the Network Level
          3. Physical Level Security
          4. Don’t Let Your Security Turn Against You
        2. TCP Wrappers
        3. Secure Shell
          1. SSH Tips and Tricks
        4. Distributing Patches, Updates, and Software Securely
      5. Developing a Backup Policy
        1. Develop a Comprehensive Backup Plan
        2. Select the Best Backup Strategy Available
          1. dump
          2. restore
          3. g4u
          4. Veritas Netbackup
          5. Legato Networker
        3. Backup Tips
          1. Do You Really Need All that Data?
          2. Keep your Backups Offsite
          3. Yes, But Can You Recover the Data?
          4. Secure Backups
      6. Summary
    3. 3. Installing and Streamlining the Cluster
      1. Setting Up the DHCP Server
        1. Installing from Source
        2. Setting DHCP Options
        3. Starting the DHCPD Server
      2. Unattended Red Hat Install with Kickstart
        1. Creating the Configuration File
        2. Booting with Kickstart
        3. Setting Up NFS for Kickstart
        4. Sample Kickstart File
      3. SystemImager
        1. Preparing SystemImager
          1. Installing Clients with SystemImager
          2. Updating Clients with SystemImager
      4. Post-Installation Procedures
      5. Summary
    4. 4. Alternative File Systems
      1. Ext2 Under a Microscope
        1. The Virtual File System Layer
        2. Playing with Blocks and Inodes
      2. Loopback File Systems
        1. Loopback with Encryption
        2. The Network Block Device
      3. The Ext3 File System
        1. Installing and Configuring Ext3
      4. ReiserFS
        1. Implementing ReiserFS
        2. Installing ReiserFS on the Root Partition
      5. The Logical Volume Manager
        1. Installing LVM
        2. Creating Disk Groups and Volumes
          1. Resizing a Volume
          2. Removing a Volume Group
          3. Backing Up with LVM: A Live Snapshot in Progress
      6. Summary
  8. II. Building Clusters
    1. 5. High Availability and Fault-Tolerant Clusters
      1. Haven’t We Heard of This Somewhere Before?
      2. Heartbeat Connectivity
        1. Attaching Hardware for Heartbeat Connectivity
        2. Configuring the Heartbeat Code
          1. Editing the ha.cf File
          2. Creating haresources
          3. Setting Up Authentication
          4. Starting Heartbeat
      3. Houston: We Have a Problem
        1. Problems with IP Connectivity
        2. Shoot The Other Machine In The Head
      4. Putting It All Together
      5. Summary
    2. 6. Load Balancing
      1. What Is This Load Balancing Thing, Anyway?
        1. Zen and the Art of Algorithms
          1. Round Robin
          2. Load-Based Balancing
      2. Balance
        1. Installation and Configuration
      3. Eddie Mission
        1. Installing Eddie
          1. Configuring Erlang
          2. Compiling the Eddie Mission
      4. Configuring the Enhanced DNS Server
        1. Configuring the HTTP Gateway
          1. Eddie Main Configuration File
          2. Eddie Gateway Configuration
          3. IP Migration Configuration
          4. Initializing Eddie
      5. Linux Virtual Server
        1. Load Balancing by any Other Method
        2. Implementing LVS
        3. Configuring the Director
          1. Setting Up LVS to Use NAT
          2. LVS NAT for Telnet
      6. Summary
    3. 7. Distributed Computing
      1. One For All, and All For One
        1. Distributed Net
        2. Search for Extraterrestrial Life
        3. Using Distributed Computing to Fight AIDS
      2. Distributed File Sharing
      3. Distributed Denial of Service
      4. Condor
        1. Installing Condor
        2. Allocating Resources with ClassAds
        3. Submitting Condor Jobs
      5. Mosix, Kernel-Based Distributed Computing
        1. Installing Mosix
        2. Configuring your Mosix Cluster
        3. Mosix at the Command Line
        4. Administrating Mosix
      6. Using Diskless Clients with Mosix
        1. Installing a Diskless Client with Mosix
      7. Summary
    4. 8. Parallel Computing
      1. Parallel Computing in a Nutshell
        1. A Beowulf by Any Other Name
        2. High-Performance Topology
      2. Cluster Installation
        1. Scyld Linux
          1. Installation of the Master Node
          2. Installation of the Slave Nodes
          3. Channel Bonding
      3. Summary
  9. III. The Care and Feeding of a Cluster
    1. 9. Programming a Parallel Cluster
      1. Coarse Granularity in a Finely Granular World
      2. Programming in a Clustered Environment
      3. MPI
        1. Getting and Installing MPI
      4. Summary
    2. 10. Cluster Management
      1. Learn to Use the Right Tools
      2. Configuring syslogd for Your Cluster
        1. Creating syslog.conf Entries
      3. General-Purpose Reporting with mon
        1. Configuring the mon Client
        2. Implementing the mon Server
      4. Big Brother Is Watching
        1. Installing and Configuring Big Brother
        2. Adding More Clients to Big Brother
        3. Using Big Brother
      5. Summary
    3. 11. Recovering When Disaster Str ikes
      1. Troubleshooting Through Layers
        1. The Physical Layer
        2. The Data-Link Layer
        3. The Network Layer
        4. The Transport and Session Layers
        5. The Presentation Layer
        6. The Application Layer
          1. Diagnosing Applications with Top
          2. Real Administrators Don’t Need a Mail Client
      2. Helpful Tools to Diagnose Your Cluster
        1. Can You View the Data?
        2. Capturing Data with tcpdump
          1. Using tcpdump to Troubleshoot Connections
          2. Filtering tcpdump Output
        3. Analyzing Packets with Ethereal
          1. Using Ethereal to Capture Data
          2. Filtering the Display
      3. When Worse Comes to Worst
        1. Reimaging from Kickstart
        2. Ghost Images
      4. This Too Shall Pass
      5. Summary
  10. IV. Appendixes
    1. A. Cluster Resources
      1. Designing a Cluster
      2. Linux File Systems
      3. Monitors
      4. High Availability Clusters
      5. Parallel Clusters
      6. Load Balanced Clusters
      7. Distributed Clusters
    2. B. Kickstart Options
      1. autostep (Optional)
      2. Authentication
      3. bootloader
      4. clearpart: Removing Partitions Based on Partition Type
      5. device
      6. deviceprobe
      7. driverdisk
      8. firewall
        1. Levels of Security
      9. install
      10. Installation Methods
      11. cdrom
      12. harddrive
      13. url
      14. interactive
      15. keyboard
      16. lang
      17. langsupport
      18. lilo
      19. lilocheck (Optional)
      20. mouse
      21. network
      22. part
      23. raid
      24. reboot (Optional)
      25. rootpw (Required)
      26. skipx (Optional)
      27. text (Optional)
      28. timezone (Required)
      29. upgrade (Optional)
      30. xconfig
      31. zerombr: Partition Table Initialization
      32. %packages: Package Selection
      33. %pre: Pre-Installation Configuration Section
      34. %post: Post-Installation Configuration Section
    3. C. DHCP Options
    4. D. Condor ClassAd Machine Attributes