IBM eX5 Implementation Guide

IBM eX5 Implementation Guide

by David Watts, Aaron Belisle, Duncan Furniss, Scott Haddow, Michael Hurman, Jeneea Jervay, Eric Kern, Cynthia Knight, Miroslav Peic, Tom Sorcic, Evans Tanurdin
Released May 2011
Publisher(s): IBM Redbooks
ISBN: None

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

High-end workloads drive ever-increasing and ever-changing constraints. In addition to requiring greater memory capacity, these workloads challenge you to do more with less and to find new ways to simplify deployment and ownership. And although higher system availability and comprehensive systems management have always been critical, they have become even more important in recent years.

Difficult challenges, such as these, create new opportunities for innovation. The IBM® eX5 portfolio delivers this innovation. This family of high-end computing introduces the fifth generation of IBM X-Architecture® technology. The family includes the IBM System x3850 X5, x3690 X5, and the IBM BladeCenter HX5. These servers are the culmination of more than a decade of x86 innovation and firsts that have changed the expectations of the industry. With this latest generation, eX5 is again leading the way as the shift toward virtualization, platform management, and energy efficiency accelerates.

This book is divided into two parts. In the first part, we provide detailed technical information about the servers in the eX5 portfolio. This information is most useful in designing, configuring, and planning to order a server solution. In the second part of the book, we provide detailed configuration and setup information to get your servers operational. We focus particularly on setting up MAX5 configurations of all three eX5 servers as well as 2-node configurations of the x3850 X5 and HX5.

This book is aimed at clients, IBM Business Partners, and IBM employees that want to understand the features and capabilities of the IBM eX5 portfolio of servers and want to learn how to install and configure the servers for use in production.

Table of contents

  1. Notices
    1. Trademarks
  2. Preface
    1. The team who wrote this book
    2. Now you can become a published author, too!
    3. Comments welcome
    4. Stay connected to IBM Redbooks
  3. Chapter 1: Introduction
    1. eX5 systems
    2. Model summary
      1. IBM System x3850 X5 models
      2. Workload-optimized x3950 X5 models
      3. x3850 X5 models with MAX5
      4. Base x3690 X5 models
      5. Workload-optimized x3690 X5 models
      6. BladeCenter HX5 models
    3. Positioning
      1. IBM System x3850 X5 and x3950 X5
      2. IBM System x3690 X5
      3. IBM BladeCenter HX5
    4. Energy efficiency
    5. Services offerings
    6. What this book contains
  4. Part 1: Product overview
  5. Chapter 2: IBM eX5 technology
    1. eX5 chip set
    2. Intel Xeon 6500 and 7500 family processors
      1. Intel Virtualization Technology
      2. Hyper-Threading Technology
      3. Turbo Boost Technology
      4. QuickPath Interconnect (QPI)
      5. Processor performance in a green world
    3. Memory
      1. Memory speed
      2. Memory DIMM placement
      3. Memory ranking
      4. Nonuniform memory architecture (NUMA)
      5. Hemisphere Mode
      6. Reliability, availability, and serviceability (RAS) features
      7. I/O hubs
    4. MAX5
    5. Scalability
    6. Partitioning
    7. UEFI system settings
      1. System power operating modes
      2. System power settings
      3. Performance-related individual system settings
    8. IBM eXFlash
      1. IBM eXFlash price-performance
    9. Integrated virtualization
      1. VMware ESXi
      2. Red Hat RHEV-H (KVM)
      3. Windows 2008 R2 Hyper-V
    10. Changes in technology demand changes in implementation
      1. Using swap files
      2. SSD drives and battery backup cache on RAID controllers
      3. Increased resources for virtualization
      4. Virtualized Memcached distributed memory caching
  6. Chapter 3: IBM System x3850 X5 and x3950 X5
    1. Product features
      1. IBM System x3850 X5 product features
      2. IBM System x3950 X5 product features
      3. IBM MAX5 memory expansion unit
      4. Comparing the x3850 X5 to the x3850 M2
    2. Target workloads
    3. Models
    4. System architecture
      1. System board
      2. QPI Wrap Card
    5. MAX5
    6. Scalability
      1. Memory scalability with MAX5
      2. Two-node scalability
    7. Processor options
    8. Memory
      1. Memory cards and DIMMs
      2. DIMM population sequence
      3. Maximizing memory performance
      4. Memory mirroring
      5. Memory sparing
      6. Effect on performance by using mirroring or sparing
    9. Storage
      1. Internal disks
      2. SAS and SSD 2.5-inch disk support
      3. IBM eXFlash and 1.8-inch SSD support
      4. SAS and SSD controllers
      5. Dedicated controller slot
      6. External storage connectivity
    10. Optical drives
    11. PCIe slots
    12. I/O cards
      1. Standard Emulex 10Gb Ethernet Adapter
      2. Optional adapters
    13. Standard onboard features
      1. Onboard Ethernet
      2. Environmental data
      3. Integrated Management Module (IMM)
      4. UEFI
      5. Integrated Trusted Platform Module (TPM)
      6. Light path diagnostics
    14. Power supplies and fans of the x3850 X5 and MAX5
      1. x3850 X5 power supplies and fans
      2. MAX5 power supplies and fans
    15. Integrated virtualization
    16. Operating system support
    17. Rack considerations
  7. Chapter 4: IBM System x3690 X5
    1. Product features
      1. System components
      2. IBM MAX5 memory expansion unit
    2. Target workloads
    3. Models
    4. System architecture
    5. MAX5
    6. Scalability
    7. Processor options
    8. Memory
      1. Memory DIMM options
      2. x3690 X5 memory population order
      3. MAX5 memory
      4. Memory balance
      5. Mixing DIMMs and the performance effect
      6. Memory mirroring
      7. Memory sparing
      8. Effect on performance of using mirroring or sparing
    9. Storage
      1. 2.5-inch SAS drive support
      2. IBM eXFlash and SSD disk support
      3. SAS and SSD controller summary
      4. Battery backup placement
      5. ServeRAID Expansion Adapter
      6. Drive combinations
      7. External SAS storage
      8. Optical drives
    10. PCIe slots
      1. Riser 1
      2. Riser 2
      3. Emulex 10Gb Ethernet Adapter
      4. I/O adapters
    11. Standard features
      1. Integrated management module
      2. Ethernet subsystem
      3. USB subsystem
      4. Integrated Trusted Platform Module
      5. Light path diagnostics
      6. Cooling
    12. Power supplies
      1. x3690 X5 power subsystem
      2. MAX5 power subsystem
    13. Integrated virtualization
    14. Supported operating systems
    15. Rack mounting
  8. Chapter 5: IBM BladeCenter HX5
    1. Introduction
      1. Comparison to the HS22 and HS22V
    2. Target workloads
    3. Chassis support
    4. Models
    5. System architecture
    6. Speed Burst Card
    7. IBM MAX5 for BladeCenter
    8. Scalability
      1. Single HX5 configuration
      2. Double-wide HX5 configuration
      3. HX5 with MAX5
    9. Processor options
    10. Memory
      1. Memory options
      2. DIMM population order
      3. Memory balance
      4. Memory mirroring
      5. Memory sparing
    11. Storage
      1. Solid-state drives (SSDs)
      2. LSI configuration utility
      3. Determining which SSD RAID configuration to choose
      4. Connecting to external SAS storage devices
    12. BladeCenter PCI Express Gen 2 Expansion Blade
    13. I/O expansion cards
      1. CIOv
      2. CFFh
    14. Standard onboard features
      1. UEFI
      2. Onboard network adapters
      3. Integrated Management Module (IMM)
      4. Video controller
      5. Trusted Platform Module (TPM)
    15. Integrated virtualization
    16. Partitioning capabilities
    17. Operating system support
  9. Part 2: Implementing scalability
  10. Chapter 6: IBM System x3850 X5 and x3950 X5
    1. Before you apply power for the first time after shipping
      1. Verify that the components are securely installed
      2. Clear CMOS memory
      3. Verify that the server completes POST before adding options
    2. Processor considerations
      1. Minimum processors required
      2. Processor operating characteristics
      3. Processor installation order
      4. Processor installation tool
    3. Local memory configuration
      1. Testing the memory DIMMs
      2. Memory fault tolerance
    4. Attaching the MAX5 memory expansion unit
      1. Before you attach the MAX5
      2. Installing in a rack
      3. MAX5 cables
      4. Accessing the DIMMs in the MAX5
    5. Forming a 2-node x3850 X5 complex
      1. Firmware requirements
      2. Processor requirements
      3. Memory requirements
      4. Cabling the servers together
    6. PCIe adapters and riser card options
      1. Generation 2 and Generation 1 PCIe adapters
      2. PCIe adapters: Slot selection
      3. Cleaning up the boot sequence
    7. Power supply considerations
    8. Using the Integrated Management Module
      1. IMM network access
      2. Configuring the IMM network interface
      3. IMM communications troubleshooting
      4. IMM functions to help you perform problem determination
    9. UEFI settings
      1. Settings needed for 1-node, 2-node, and MAX5 configurations
      2. UEFI performance tuning
    10. Installing an OS
      1. Installing without a local optical drive
      2. Use of embedded VMware ESXi
      3. Installing the ESX 4.1 or ESXi 4.1 Installable onto x3850 X5 (1/3)
      4. Installing the ESX 4.1 or ESXi 4.1 Installable onto x3850 X5 (2/3)
      5. Installing the ESX 4.1 or ESXi 4.1 Installable onto x3850 X5 (3/3)
      6. OS installation tips and instructions on the web
      7. Downloads and fixes for x3850 X5 and MAX5
      8. SAN storage reference and considerations
    11. Failure detection and recovery
      1. What happens when a node fails or the MAX5 fails
      2. Reinserting the QPI wrap cards for extended outages
      3. Tools to aid hardware troubleshooting for x3850 X5
      4. Recovery process
  11. Chapter 7: IBM System x3690 X5
    1. Before you apply power for the first time after shipping
      1. Verify that the components are securely installed
      2. Clear CMOS memory
      3. Verify that the server will complete POST before adding options
    2. Processor considerations
      1. Minimum processors required
      2. Processor operating characteristics
    3. Memory considerations
      1. Local memory installation considerations
      2. Testing the memory DIMMs
      3. Memory fault tolerance
    4. MAX5 considerations
      1. Before you attach the MAX5
      2. Installing in a rack
      3. MAX5 cables
      4. Accessing the DIMMs in the MAX5
    5. PCIe adapters and riser card options
      1. Generation 2 and Generation 1 PCIe adapters
      2. PCIe adapters: Slot selection
      3. Cleaning up the boot sequence
    6. Power supply considerations
    7. Using the Integrated Management Module
      1. IMM network access
      2. Configuring the IMM network interface
      3. IMM communications troubleshooting
      4. IMM functions to help you perform problem determination (1/2)
      5. IMM functions to help you perform problem determination (2/2)
    8. UEFI settings
      1. Scaled system settings
      2. Operating system-specific settings
      3. Power and performance system settings
      4. Optimizing boot options
    9. Operating system installation
      1. Installation media
      2. Integrated virtualization hypervisor
      3. Windows Server 2008 R2
      4. Red Hat Enterprise Linux 6 and SUSE Linux Enterprise Server 11
      5. VMware vSphere ESXi 4.1
      6. VMware vSphere ESX 4.1
      7. Downloads and fixes for the x3690 X5 and MAX5
      8. SAN storage reference and considerations
    10. Failure detection and recovery
      1. System alerts
      2. System recovery
  12. Chapter 8: IBM BladeCenter HX5
    1. Before you apply power for the first time after shipping
      1. Verifying that the components are securely installed
      2. Clearing CMOS memory
      3. Verifying the server boots before adding options
    2. Planning to scale: Prerequisites
      1. Processors supported and requirements to scale
      2. Minimum memory requirement
      3. Required firmware of each blade and the AMM
    3. Recommendations
      1. Power sharing cap
      2. BladeCenter H considerations
    4. Local storage considerations and array setup
      1. Launching the LSI Setup Utility
      2. Creating a RAID-1 mirror using the LSI Setup Utility
      3. Using IBM ServerGuide to configure the LSI controller
      4. Speed Burst Card reinstallation
    5. UEFI settings
      1. UEFI performance tuning
      2. Start-up parameters
      3. HX5 single-node UEFI settings
      4. HX5 2-node UEFI settings
      5. HX5 with MAX5 attached
      6. Operating system-specific settings in UEFI
    6. Creating an HX5 scalable complex
      1. Troubleshooting HX5 problems
    7. Operating system installation
      1. Operating system installation media
      2. VMware ESXi on a USB key (1/2)
      3. VMware ESXi on a USB key (2/2)
      4. Installing ESX 4.1 or ESXi 4.1 Installable onto HX5 (1/3)
      5. Installing ESX 4.1 or ESXi 4.1 Installable onto HX5 (2/3)
      6. Installing ESX 4.1 or ESXi 4.1 Installable onto HX5 (3/3)
      7. Windows installation tips and settings
      8. Red Hat Enterprise Linux installation tips and settings
      9. SUSE Linux Enterprise Server installation tips and settings
      10. Downloads and fixes for HX5 and MAX5
      11. SAN storage reference and considerations
    8. Failure detection and recovery
      1. Tools to aid hardware troubleshooting for the HX5
      2. Reinserting the Speed Burst card for extended outages
      3. Effects of power loss on HX5 2-node or MAX5 configurations
  13. Chapter 9: Management
    1. Introduction
    2. Integrated Management Module (IMM)
      1. IMM out-of-band configuration
      2. IMM in-band configuration
      3. Updating firmware using the IMM
    3. Advanced Management Module (AMM)
      1. Accessing the Advanced Management Module
      2. Service Advisor
      3. Updating firmware using the AMM
    4. Remote control
      1. Accessing the Remote Control feature on the x3690 X5 and the x3850 X5
      2. Accessing the Remote Control feature for the HX5
    5. IBM Systems Director 6.2
      1. Discovering the IMM of a single-node x3690 X5 or x3850 X5 out-of-band via IBM Systems Director
      2. Discovering a 2-node x3850 X5 via IBM Systems Director 6.2.x
      3. Discovering a single-node HX5 via IBM Systems Director
      4. Discovering a 2-node HX5 via IBM Systems Director 6.2.x
      5. Service and Support Manager (1/2)
      6. Service and Support Manager (2/2)
      7. Performing tasks against a 2-node system
    6. IBM Electronic Services
    7. Advanced Settings Utility (ASU)
      1. Using ASU to configure settings in IMM-based servers
      2. Common problems
      3. Command examples
    8. IBM ServerGuide (1/2)
    9. IBM ServerGuide (2/2)
    10. IBM ServerGuide Scripting Toolkit
    11. Firmware update tools and methods
      1. Configuring UEFI
      2. Requirements for updating scalable systems
      3. IBM Systems Director
    12. UpdateXpress System Pack Installer
    13. Bootable Media Creator (1/2)
    14. Bootable Media Creator (2/2)
    15. MegaRAID Storage Manager
      1. Installation
      2. Drive states
      3. Virtual drive states
      4. MegaCLI utility for storage management
    16. Serial over LAN
      1. Enabling SoL in UEFI
      2. BladeCenter requirements
      3. Enabling SoL in the operating system
      4. How to start a SoL connection
  14. Abbreviations and acronyms
  15. Related publications
    1. IBM Redbooks publications
    2. Other publications
    3. Online resources
    4. Help from IBM
  16. Index (1/3)
  17. Index (2/3)
  18. Index (3/3)
  19. Back cover

Product information

  • Title: IBM eX5 Implementation Guide
  • Author(s): David Watts, Aaron Belisle, Duncan Furniss, Scott Haddow, Michael Hurman, Jeneea Jervay, Eric Kern, Cynthia Knight, Miroslav Peic, Tom Sorcic, Evans Tanurdin
  • Release date: May 2011
  • Publisher(s): IBM Redbooks
  • ISBN: None