IBM Spectrum Scale and IBM Elastic Storage System Network Guide

IBM Spectrum Scale and IBM Elastic Storage System Network Guide

by Kedar Karmarkar, John Lewars, Sandeep R. Patil, Sandeep Naik, Kevin Gildea, Rakesh Chutke, Larry Coyne
Released February 2021
Publisher(s): IBM Redbooks
ISBN: 9780738459417

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

High-speed I/O workloads are moving away from the SAN to Ethernet and IBM® Spectrum Scale is pushing the network limits. The IBM Spectrum® Scale team discovered that many infrastructure Ethernet networks that were used for years to support various applications are not designed to provide a high-performance data path concurrently to many clients from many servers.

IBM Spectrum Scale is not the first product to use Ethernet for storage access. Technologies, such as Fibre Channel over Ethernet (FCoE), scale out NAS, and IP connected storage (iSCSI and others) use Ethernet though IBM Spectrum Scale as the leader in parallel I/O performance, which provides the best performance and value when used on a high-performance network. This IBM Redpaper publication is based on lessons that were learned in the field by deploying IBM Spectrum Scale on Ethernet and InfiniBand networks.

This IBM Redpaper® publication answers several questions, such as, "How can I prepare my network for high performance storage?", "How do I know when I am ready?", and "How can I tell what is wrong?" when deploying IBM Spectrum Scale and IBM Elastic Storage® Server (ESS).

This document can help IT architects get the design correct from the beginning of the process. It also can help the IBM Spectrum Scale administrator work effectively with the networking team to quickly resolve issues.

Table of contents

  1. Front cover
  2. Notices
    1. Trademarks
  3. Preface
    1. Authors
    2. Now you can become a published author, too!
    3. Comments welcome
    4. Stay connected to IBM Redbooks
  4. Chapter 1. IBM Spectrum Scale introduction
    1. 1.1 How is IBM Spectrum Scale different from other network storage technologies
    2. 1.2 Types of network traffic in IBM Spectrum Scale
      1. 1.2.1 Control traffic
      2. 1.2.2 Data traffic
      3. 1.2.3 Administration traffic
    3. 1.3 High-level network flow for data and control traffic
      1. 1.3.1 IBM Spectrum Scale supported networks
      2. 1.3.2 IBM Spectrum Scale expels
  5. Chapter 2. Network planning and best practices for IBM Spectrum Scale and IBM Elastic Storage System
    1. 2.1 Network planning and sizing
    2. 2.2 Bandwidth requirements
      1. 2.2.1 Aggregate cluster bandwidth
      2. 2.2.2 Single node bandwidth
    3. 2.3 Deciding network bandwidth planning
    4. 2.4 Network design considerations
    5. 2.5 OSI Layer
      1. 2.5.1 OSI Layer 1: Physical layer
      2. 2.5.2 OSI Layer 2: Link layer
      3. 2.5.3 OSI Layer 3+4
    6. 2.6 Network multipathing
      1. 2.6.1 Layer 2 multipathing
      2. 2.6.2 Hosts-to-switch side
      3. 2.6.3 Switch side
      4. 2.6.4 Layer 3 multipathing
    7. 2.7 Network infrastructure
      1. 2.7.1 Common network topologies
      2. 2.7.2 LAN and WAN considerations
      3. 2.7.3 Network monitoring tools
      4. 2.7.4 Flow control techniques
    8. 2.8 Network considerations for special use cases
  6. Chapter 3. Implementation recommendations
    1. 3.1 Network verification tools
      1. 3.1.1 IBM Spectrum Scale mmnetverify command
      2. 3.1.2 IBM Spectrum Scale nsdperf tool
    2. 3.2 Ethernet Network Adaptor configuration settings
      1. 3.2.1 Ring buffers
      2. 3.2.2 Maximum transmission unit
      3. 3.2.3 Transmission queue length
      4. 3.2.4 Flow control
    3. 3.3 TCP/IP settings
      1. 3.3.1 TCP/IP send and receive buffers
      2. 3.3.2 TCP/IP window scaling
      3. 3.3.3 TCP/IP settings for network with packet loss
      4. 3.3.4 Other TCP/IP setting
    4. 3.4 Remote Direct Memory Access settings
      1. 3.4.1 Settings to enable RDMA between nodes
      2. 3.4.2 Tuning for RDMA
    5. 3.5 Network multipathing settings
  7. Chapter 4. Network monitoring and troubleshooting
    1. 4.1 Network-related issues overview
    2. 4.2 Network monitoring with IBM Spectrum Scale GUI
      1. 4.2.1 Monitoring Network Events Using the GUI
      2. 4.2.2 Monitoring Network Performance Using the GUI
      3. 4.2.3 Network Monitoring with IBM Spectrum Scale mmhealth command
      4. 4.2.4 IBM Spectrum Scale call back commands to monitor network-related events
    3. 4.3 Network troubleshooting with IBM Spectrum Scale mmdiag command
    4. 4.4 Verifying network with IBM Spectrum Scale mmnetverify command
    5. 4.5 Testing network performance with IBM Spectrum Scale nsdperf tool
    6. 4.6 IBM Spectrum Scale mmuserauth check
    7. 4.7 Using IBM Spectrum Scale log messages to debug network-related issues
    8. 4.8 Node expels because of network issues
    9. 4.9 Understanding and resolving the effect of network packet loss
      1. 4.9.1 Overview and effect
    10. 4.10 Resource issues
  8. Appendix A. IBM Spectrum Scale lease configuration variables effect on expel timing flows
    1. Lease configuration parameters related to disk leasing
  9. Appendix B. nsdperf command examples
    1. Single Client and Single Server
    2. Multiple clients and servers
    3. Supplemental nsdperf tests and commands
  10. Related publications
    1. IBM Redbooks
    2. Online resources
    3. Help from IBM
  11. Back cover

Product information

  • Title: IBM Spectrum Scale and IBM Elastic Storage System Network Guide
  • Author(s): Kedar Karmarkar, John Lewars, Sandeep R. Patil, Sandeep Naik, Kevin Gildea, Rakesh Chutke, Larry Coyne
  • Release date: February 2021
  • Publisher(s): IBM Redbooks
  • ISBN: 9780738459417