O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Storage Design and Implementation in vSphere 6: A Technology Deep Dive, 2nd Edition

Book Description

Now fully updated: The authoritative, comprehensive guide to vSphere 6 storage implementation and management

Effective VMware virtualization storage planning and management has become crucial—but it can be extremely complex. Now, VMware’s leading storage expert thoroughly demystifies the “black box” of vSphere 6 storage and provides illustrated, step-by-step procedures for performing every key task associated with it. Mostafa Khalil presents techniques based on years of personal experience helping customers troubleshoot storage in their vSphere production environments. Drawing on more experience than anyone else in the field, he combines expert guidelines, insights for better architectural design, best practices for planning and management, common configuration details, and deep dives into both vSphere and third-party storage. Storage Design and Implementation in vSphere 6, Second Edition will give you the deep understanding you need to make better upfront storage decisions, quickly solve problems if they arise, and keep them from occurring in the first place.

Coverage includes:

  • Planning and implementing Fibre Channel, FCoE, and iSCSI storage in vSphere virtualized environments
  • Implementing vSphere Pluggable Storage Architecture native multipathing, SATP, PSP, plug-ins, rules, registration, and more
  • Working with Active/Passive and Pseudo-Active/Active ALUA SCSI-3 storage arrays
  • Maximizing availability with multipathing and failover
  • Improving efficiency and value by unifying and centrally managing heterogeneous storage configurations
  • Understanding Storage Virtualization Devices (SVDs) and designing storage to take advantage of them
  • Implementing VMware Virtual Machine File System (VMFS) to maximize performance and resource utilization
  • Working with virtual disks and raw device mappings (RDMs)
  • Managing snapshots in VMFS and Virtual Volumes environments
  • Implementing and administering NFS, VAAI, Storage vMotion, VisorFS, and VASA
  • Integrating VSAN core and advanced features
  • Using Virtual Volumes to streamline storage operations and gain finer VM-level control over external storage

Table of Contents

  1. About This E-Book
  2. Title Page
  3. Copyright Page
  4. Dedication Page
  5. Contents at a glance
  6. Contents
  7. Introduction
  8. Part I Storage Protocols and Block Devices
    1. 1 Storage Types
      1. History of Storage
        1. Birth of the Hard Disks
        2. Along Comes SCSI
        3. PATA and SATA—SCSI’s Distant Cousins?
        4. Measuring Storage Capacity
        5. Permanent Storage Media Relevant to vSphere 6
      2. Summary
    2. 2 Fibre Channel Storage Connectivity
      1. SCSI Standards and Protocols
        1. SCSI-2 and SCSI-3 Standards
        2. Fibre Channel Protocol
        3. Decoding an EMC Symmetrix WWPN
        4. Locating a Target’s WWNN and WWPN Seen by ESXi 6 Hosts
        5. SAN Topology
        6. Fabric Switches
        7. FC Zoning
        8. Designing Storage with No Single Points of Failure
      2. Summary
    3. 3 FCoE Storage Connectivity
      1. FCoE (Fibre Channel over Ethernet)
      2. FCoE Initialization Protocol
      3. FCoE Initiators
        1. Hardware FCoE Adapters
        2. Software FCoE Adapters
      4. Overcoming Ethernet Limitations
        1. Flow Control in FCoE
      5. Protocols Required for FCoE
        1. Priority-Based Flow Control
        2. Enhanced Transmission Selection
        3. Data Center Bridging Exchange
        4. 10GigE—A Large Pipeline
      6. 802.1p Tag
      7. Hardware FCoE Adapters
      8. Configuring HW FCoE Adapters
      9. Implementing SW FCoE in ESXi 6
      10. Configuring SW FCoE Network Connections
      11. Enabling a Software FCoE Adapter
      12. Removing or Disabling a Software FCoE Adapter
        1. Using the UI to Remove a Software FCoE Adapter
        2. Using the CLI to Remove a Software FCoE Adapter
      13. Troubleshooting FCoE
        1. ESXCLI
        2. FCoE-Related Logs
        3. How to Verify SW FCoE Connectivity
        4. Proc Nodes Can Be Useful, Too
      14. Parting Tips
      15. Summary
    4. 4 iSCSI Storage Connectivity
      1. iSCSI Protocol
        1. Customizing Storage Network Configuration
        2. Troubleshooting iSCSI Connectivity
      2. Summary
    5. 5 vSphere Pluggable Storage Architecture
      1. Native Multipathing
      2. Storage Array Type Plug-in (SATP)
        1. Listing SATPs on an ESXi 6 Host
      3. Path Selection Plug-in (PSP)
        1. Listing PSPs on an ESXi 6 Host
      4. Third-Party Plug-ins
      5. Multipathing Plug-ins (MPPs)
      6. Anatomy of PSA Components
      7. I/O Flow Through PSA and NMP
        1. Classification of Arrays Based on How They Handle I/O
        2. Paths and Path States
        3. Preferred Path Setting
        4. Flow of I/O Through NMP
      8. Listing Multipath Details
        1. Listing Paths to a LUN by Using the UI
        2. Listing Paths to a LUN by Using the CLI
        3. Identifying Path States and on Which Path the I/O Is Sent—FC
        4. Example of Listing Paths to an iSCSI-Attached Device
        5. Identifying Path States and on Which Path the I/O Is Sent—iSCSI
        6. Example of Listing Paths to an FCoE-Attached Device
        7. Identifying Path States and on Which Path the I/O Is Sent—FC
      9. Claim Rules
      10. MP Claim Rules
      11. Plug-in Registration
      12. SATP Claim Rules
      13. Modifying PSA Plug-in Configurations Using the UI
        1. Which PSA Configurations Can Be Modified Using the UI?
      14. Modifying PSA Plug-ins Using the CLI
        1. Available CLI Tools and Their Options
        2. Adding a PSA Claim Rule
        3. Deleting a Claim Rule
        4. Masking Paths to a Certain LUN
        5. Unmasking a LUN
        6. Changing PSP Assignment via the CLI
      15. Summary
    6. 6 ALUA
      1. ALUA Definition
        1. ALUA Target Port Groups
        2. Asymmetric Access State
        3. ALUA Management Modes
        4. ALUA Followover
        5. Identifying Device ALUA Configuration
        6. Troubleshooting ALUA
      2. Summary
    7. 7 Multipathing and Failover
      1. What Is a Path?
      2. Where Is the Active Path?
        1. Identifying the Current Path Using the CLI
        2. Identifying the I/O (Current) Path Using the UI
      3. LUN Discovery and Path Enumeration
      4. Sample LUN Discovery and Path Enumeration Log Entries
      5. Factors Affecting Multipathing
        1. Accessing Advanced Options
      6. Failover Triggers
        1. SCSI Sense Codes
        2. Path Failover Triggers
      7. Path States
        1. Factors Affecting Paths States
      8. Path Selection Plug-ins
        1. VMW_PSP_FIXED
        2. VMW_PSP_MRU
        3. VMW_PSP_RR
      9. When and How to Change the Default PSP
        1. When Should You Change the Default PSP?
        2. How to Change the Default PSP
      10. PDL and APD
        1. Unmounting a VMFS Volume
        2. Detaching a Device Whose Datastore Was Unmounted
      11. Path Ranking
        1. Path Ranking for ALUA and Non-ALUA Storage
        2. How Does Path Ranking Work for ALUA Arrays?
        3. How Does Path Ranking Work for Non-ALUA Arrays?
        4. Configuring Ranked Paths
      12. Summary
    8. 8 Third-Party Multipathing I/O Plug-ins
      1. MPIO Implementations on vSphere 6
      2. EMC PowerPath/VE 6
        1. Downloading PowerPath/VE
        2. PowerPath/VE Installation Overview
        3. What Gets Installed?
        4. Installing PowerPath/VE by Using the Local CLI
        5. Verifying the PowerPath/VE Installation
        6. Listing Devices Claimed by PowerPath/VE
        7. Managing PowerPath/VE
        8. Uninstalling PowerPath/VE
      3. Hitachi Dynamic Link Manager (HDLM)
        1. Obtaining Installation Files
        2. Installing HDLM
        3. Modifying HDLM PSP Assignments
        4. Locating Certified Storage on the VMware HCL
        5. Downloading Documentation
        6. Downloading the Installation File and the Setup Script
        7. How Does EQL MEM Work?
        8. Installing EQL MEM on vSphere 6
        9. Uninstalling the Dell PSP EQL Routed MEM
      4. Summary
    9. 9 Using Heterogeneous Storage Configurations
      1. What Is a “Heterogeneous” Storage Environment?
      2. Heterogeneous Storage Scenarios
      3. ESXi 6 View of Heterogeneous Storage
        1. Basic Rules of Using Heterogeneous Storage
        2. Naming Conventions
        3. How Does All This Fit Together?
      4. Summary
    10. 10 Using VMDirectPath I/O
      1. What Is VMDirectPath?
      2. Which I/O Devices Are Supported?
        1. Locating Hosts Supporting VMDirectPath I/O on the HCL
      3. Configuring VMDirectPath I/O on vSphere 6.0
        1. What Gets Added to the VM’s Configuration File?
      4. Toggling an I/O Adapter as Passthrough on vSphere 6.5
        1. Adding a Passthrough PCI Device to a VM by Using vSphere Web Client
        2. Adding a Passthrough PCI Device to a VM by Using the HTML5 ESXi Host Client
      5. Practical Examples of VM Design Scenarios Using VMDirectPath I/O
        1. Hyperconverged Infrastructure
        2. Passing Through Physical Tape Devices
        3. Supported VMDirectPath I/O Devices
        4. DirectPath I/O Example
      6. Troubleshooting VMDirectPath I/O
        1. Interrupt Handling and IRQ Sharing
        2. Device Sharing
      7. Summary
    11. 11 Storage Virtualization Devices (SVDs)
      1. The SVD Concept
        1. How Does It Work?
        2. Constraints
      2. SVD Design Decisions
        1. Front-End Design Choices
        2. Back-end Design Choices
        3. LUN Presentation Considerations
        4. RDM Considerations
      3. Summary
  9. Part II File Systems
    1. 12 VMFS Architecture
      1. History of VMFS
        1. VMFS3 on-Disk Layout
        2. VMFS5 Layout
        3. Common Causes of Partition Table Problems
        4. Re-creating a Lost Partition Table for VMFS3 Datastores
        5. Re-creating a Lost Partition Table for VMFS5 Datastores
        6. More About VOMA
        7. Re-creating the Partition Table
        8. Preparing for the Worst: Recovering from a File System Corruption
        9. Span or Grow?
        10. Upgrading to VMFS5
        11. VMFS6
      2. Summary
    2. 13 Virtual Disks and RDMs
      1. The Big Picture
      2. Virtual Disks
        1. Virtual Disk Types
        2. Thin-on-Thin Configuration
        3. Virtual Disk Modes
      3. Creating Virtual Disks Using the UI
        1. Creating Virtual Disks During VM Creation
        2. Creating a Virtual Disk After VM Creation
      4. Creating Virtual Disks Using vmkfstools
        1. Creating a Zeroed Thick Virtual Disk Using vmkfstools
        2. Creating an Eager Zeroed Thick Virtual Disk Using vmkfstools
        3. Creating a Thin Virtual Disk Using vmkfstools
        4. Cloning Virtual Disks Using vmkfstools
      5. Raw Device Mappings
        1. Creating Virtual Mode RDMs Using the UI
        2. Creating Physical Mode RDMs Using the UI
        3. Creating RDMs Using the Command-Line Interface
      6. Listing RDM Properties
        1. Listing RDM Properties Using vmkfstools
        2. Listing RDM Properties Using the UI
      7. Virtual Storage Adapters
        1. Selecting the Type of Virtual Storage Adapter
        2. VMware Paravirtual SCSI Controller
        3. SCSI Bus Sharing
      8. Virtual Machine Snapshots
        1. Creating the VM’s First Snapshot While the VM Is Powered Off
        2. Creating a VM Second Snapshot While Powered On
      9. Snapshot Operations
        1. Reverting to a Snapshot
        2. Deleting a Snapshot
        3. Consolidating Snapshots
      10. Reverting to the Latest Snapshot
      11. Linked Clones
      12. Summary
    3. 14 Distributed Locks
      1. Basic Locking
        1. Using ATS for the Heartbeat
        2. What Happens When a Host Crashes?
      2. Optimistic Locking
        1. Dynamic Resource Allocation
        2. SAN Aware Retries
        3. Optimistic I/O
      3. Operations That Require SCSI Reservations
        1. MSCS/WSFC-Related SCSI Reservations
        2. Perennial Reservations
      4. Under the Hood of Distributed Locks
      5. Summary
    4. 15 Snapshot Handling
      1. What Is a Snapshot?
      2. What Is a Replica?
      3. What Is a Mirror?
      4. VMFS Signature
        1. Listing Datastores’ UUIDs via the CLI
      5. Effects of Snapshots on VMFS Signatures
      6. How to Handle VMFS Datastores on Snapshot LUNs
      7. Resignaturing Datastores
        1. Resignaturing a VMFS Datastore Using vSphere 6 Web Client
        2. Resignaturing a VMFS Datastore Using ESXCLI
      8. Force-Mounting
        1. Force-Mounting a VMFS Snapshot Using vSphere 6 Web Client
        2. Force-Mounting a VMFS Snapshot Using ESXCLI
      9. Sample Script to Force-Mount All Snapshots on Hosts in a Cluster
      10. Summary
    5. 16 NFS
      1. History of NFS
        1. NFS on vSphere 6
      2. Configuring an NFS Volume on NAS Servers
        1. NFS Exports on EMC Unity
        2. Configuring Replication on EMC Unity
        3. Creating an NFSv3 Storage VM (SVM) on a NetApp Filer
        4. Creating an NFSv3 Datastore on a NetApp Filer
        5. Using Kerberos Authentication with NFSv4.1 Datastores
        6. ESXi Kerberos 5 Components
      3. Configuring a NetApp Filer for NFSv4.1
        1. Creating an NFSv4.1 Storage VM (SVM) on a NetApp Filer by Using the CLI
        2. Creating an NFSv4.1 Storage VM (SVM) on a NetApp Filer by Using the UI
        3. Preparing ESXi Hosts for Kerberos Authentication
        4. Configuring ESXi Hosts for Kerberos Authentication
        5. Preparing AD for NFSv4.1 Kerberos Authentication
        6. Configuring AD for UNIX Identity Management
        7. Creating Kerberos-to-UNIX User Name Mapping
        8. Creating an NFSv4.1 Datastore on a NetApp Filer
        9. NFS Locking Mechanism
        10. NFS Client Advanced Configurations
        11. Useful Authentication-Related Commands
        12. Troubleshooting NFS Connectivity
      4. Summary
    6. 17 VAAI
      1. What Is VAAI?
      2. VAAI Primitives
        1. Hardware Acceleration APIs
        2. Thin Provisioning APIs
      3. Full Copy Primitive (XCOPY)
      4. Block Zeroing Primitive (WRITE_SAME)
      5. Hardware Accelerated Locking Primitive (ATS)
        1. ATS Enhancements on VMFS5
      6. Thin Provisioned APIs
      7. NAS VAAI Primitives
      8. Enabling and Disabling VAAI Primitives
        1. Disabling Block Device Primitives by Using the UI
        2. Disabling Block Device VAAI Primitives by Using the CLI
        3. Disabling the UNMAP Primitive by Using the CLI
        4. Disabling NAS VAAI Primitives
      9. VAAI Filter and VAAI Plug-in
        1. Locating Supported VAAI-Capable Block Devices
        2. Locating Supported VAAI-Capable NAS Devices
      10. Listing Registered Filter and VAAI Plug-ins
      11. Listing the Configurations of VAAI Filters and Plug-ins
      12. XCOPY Custom Options
        1. Configuring XCOPY Custom Options
      13. Listing VAAI VMkernel Modules
      14. Identifying VAAI Primitives Supported by a Device
        1. Listing Block Device VAAI Support Status by Using the CLI
        2. Listing NAS Device VAAI Support Status
        3. Listing VAAI Support Status by Using the UI
      15. Displaying Block Device VAAI I/O Stats Using esxtop
      16. The VAAI T10 Standard Commands
      17. Troubleshooting VAAI Primitives
        1. Example Using the VMFS Volume Label
        2. Example Using VMFS Volume UUID
        3. Sample Log Entries Related to VAAI
      18. Summary
    7. 18 Storage vMotion
      1. Storage vMotion History
      2. Storage vMotion Requirements
      3. Storage vMotion Within the Same Host
      4. Storage vMotion Between Hosts’ Local Datastores
      5. How Does SvMotion Actually Work?
      6. Summary
    8. 19 VisorFS
      1. What Is VisorFS?
      2. RAM Disks
      3. Stateful or Stateless?
      4. Boot Device Partitions
      5. Rolling Back to a Previous Version
      6. Summary
    9. 20 VASA
      1. What Is VASA?
        1. VASA Architecture
        2. VASA Versions
        3. Configuring the vSphere Environment to Use VASA
        4. Locating VASA-Certified Storage Arrays on vCG
        5. Installing VASA Providers
      2. Registering VASA Providers
      3. Summary
    10. 21 vSAN Core Features
      1. What Is vSAN?
      2. vSAN Requirements
      3. The File System on the vSAN Datastore
      4. vSAN Network
        1. Configuring vDS
      5. Solid State Drives and Flash Storage
      6. I/O Adapters
      7. VMware Certification Guide (vCG)
        1. Looking Up vSAN Ready Nodes on the vCG
      8. Ruby vSphere Console (RVC)
      9. vSAN Software Components
      10. Creating and Configuring a vSAN Cluster
      11. vSAN Disk Groups
      12. vSAN Storage Policies
        1. vSAN Storage Policy Placement Rules
        2. Erasure Coding (RAID 5 and RAID 6)
        3. Reconfiguring Existing Virtual Disks’ Storage Policies
      13. HA on vSAN
      14. vSAN Easy Installation
      15. Summary
    11. 22 vSAN Advanced Features
      1. Fault Domains
        1. Creating vSAN Fault Domains
        2. Listing Fault Domains via the CLI
        3. Component Placement on Fault Domains
      2. Stretched Clusters
        1. A vSAN Stretched Cluster Inter-data Site Network
        2. Calculating a vSAN Network’s Minimum Bandwidth Requirement
        3. vSAN Stretched Cluster Witness
        4. vSAN Stretched Cluster Witness-to-Data-Sites Network
        5. Deploying a Witness Virtual Appliance
        6. Customizing the Nested ESXi Witness VM Network
        7. Adding a Nested ESXi Witness Node to a Datacenter
        8. Verifying Witness Network Configuration
        9. Configuring a Stretched Cluster
        10. Using RVC to List Stretched Cluster Witness Information
        11. vSAN Stretched Cluster I/O Path
        12. vSAN Stretched Cluster Failure Modes
        13. Sample UI and cmmds-tool Output from vSAN Stretched Cluster Failures
        14. RAID 1 Examples
        15. RAID 10 Examples
      3. Comparison Between Fault Domains and Stretched Clusters
      4. Nested vSAN Fault Domains
        1. Nested FD Storage Policies
        2. Configuring a Nested FD Policy
        3. Nested RAID 1 Configuration
        4. Nested RAID 10 Configuration
        5. Nested Fault Domain I/O Flow
      5. Remote Office/Branch Office (ROBO) vSAN Cluster
        1. Configuring a ROBO Cluster
      6. Deduplication and Compression
        1. Deduplication and Compression Licensing
        2. Configuring Deduplication and Compression
        3. Displaying Dedupe and Compression Stats
      7. The vSAN Health Check UI
        1. Configuring Health Check
        2. Accessing the Health Check UI
        3. Health Check Tests
        4. Running Burn-in Tests
      8. vSAN Performance Monitoring
        1. Enabling vSAN Performance Service
        2. Accessing vSAN Performance Charts in the UI
        3. Running vSAN Performance Diagnostics
        4. Displaying Cluster Back-End Performance Charts
        5. Displaying Cluster-Level VM Consumption Charts
        6. Displaying Host-Level vSAN Performance Charts
        7. Displaying VM-Level vSAN Performance Charts
      9. vSAN Observer
        1. Starting vSAN Observer for Live Monitoring
        2. Collecting an Observer Bundle
        3. Browsing vSAN Observer HTML Bundle Stats
      10. vSAN iSCSI Targets (VITs)
        1. vSAN iSCSI Targets Architecture
        2. Configuring a VIT Network
        3. Enabling VITs
        4. Creating VITs
        5. Creating Initiator Groups
        6. VIT Configuration Under the Hood
        7. Monitoring VIT
        8. Using ESCXLI to List VITs details
      11. Storage Policy Noncompliance
        1. Monitoring Resyncing Components via the UI
        2. Monitoring Resyncing Components via the RVC
      12. Resync Throttling
        1. Resync Throttling via the UI
        2. Resync Throttling via the CLI
        3. Sample Object Properties During Resync
      13. vSAN Sparse Format
      14. Common vSAN Basic Administrative Tasks
        1. Removing a Host from a vSAN Cluster
        2. Rejoining a Host to a vSAN Cluster
        3. Using cmmds_tool to List Various Types
        4. Listing a Disk’s Stats by Using the CLI
        5. vSphere 6.5 vSAN Health CLI Namespace
      15. Summary
    12. 23 Virtual Volumes (VVols)
      1. What Are VVols?
        1. VVols Architecture
        2. Configuring the vSphere Environment to Use VVols
        3. Locating VASA-Certified Storage Arrays on the vCG
        4. Installing VASA Providers
      2. Configuring Block Storage Arrays for VVols
        1. Configuring Dell Virtual Storage Manager (VSM) and Registering the VASA Provider with vCenter
      3. Allowing ESXi Hosts’ iSCSI Initiators to Access EQL VVols Protocol Endpoints
      4. Configuring a NAS Server for VVols
      5. Creating Storage Pools for Use by EMC Unity VVols SCs
      6. Configuring the vSphere 6 Environment for VVols
      7. Creating Virtual Machine Storage Policies for VVols
      8. Deploying New VMs Using Storage Policies
      9. I/O Flow to VVols
      10. Listing Virtual Machine Files on a VVols Datastore via the CLI
      11. OSFS and Its Role in VVols
      12. Storage Array’s View of VVols
      13. Native Snapshots on VVols
      14. Putting It All Together
      15. VVols Interoperability with Other vSphere Features
      16. VVols Replication
      17. Summary
  10. Index
  11. Code Snippets