Mastering Proxmox - Third Edition

Mastering Proxmox - Third Edition

by Wasim Ahmed
Released November 2017
Publisher(s): Packt Publishing
ISBN: 9781788397605

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Discover real world scenarios for Proxmox troubleshooting and become an expert cloud builder

About This Book

  • Formulate Proxmox-based solutions and set up virtual machines of any size while gaining expertise even on the most complex multi-cluster setups
  • Master the skills needed to analyze, monitor, and troubleshoot real-world virtual environments
  • This is the most up-to-date title on mastering Proxmox, with examples based on the new Linux Kernel 4.10.15 and Debian Stretch (9.x)

Who This Book Is For

This book is for Linux and system administrators and professionals working in IT teams who would like to design and implement an enterprise-quality virtualized environment using Proxmox. Some knowledge of networking and virtualization concepts is assumed.

What You Will Learn

  • Install basic Proxmox VE nodes and get to know the Proxmox GUI
  • Get to know Proxmox's internal structure and mechanics
  • Create and manage KVM or LXC-based virtual machines
  • Understand advanced virtual networks
  • Configure high availability Proxmox nodes
  • Integrate Ceph big data storage with the Proxmox hypervisor
  • Plan a large virtual environment for cloud-based services
  • Discover real-world scenarios for Proxmox troubleshooting

In Detail

Proxmox is an open source server virtualization solution that has enterprise-class features for managing virtual machines, for storage, and to virtualize both Linux and Windows application workloads.

You'll begin with a refresher on the advanced installation features and the Proxmox GUI to familiarize yourself with the Proxmox VE hypervisor. Then, you'll move on to explore Proxmox under the hood, focusing on storage systems, such as Ceph, used with Proxmox. Moving on, you'll learn to manage KVM virtual machines, deploy Linux containers fast, and see how networking is handled in Proxmox. You'll also learn how to protect a cluster or a VM with a firewall and explore the new high availability features introduced in Proxmox VE 5.0. Next, you'll dive deeper into the backup/restore strategy and see how to properly update and upgrade a Proxmox node. Later, you'll learn how to monitor a Proxmox cluster and all of its components using Zabbix. Finally, you'll discover how to recover Promox from disaster strikes through some real-world examples.

By the end of the book, you'll be an expert at making Proxmox work in production environments with minimal downtime.

Style and approach

This book walks you through every aspect of virtualization using Proxmox using a practical, scenario-based approach that features best practices and all the weaponry you need to succeed when building virtual environments with Proxmox 5.0.

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Conventions
    5. Reader feedback
    6. Customer support
      1. Downloading the color images of this book
      2. Errata
      3. Piracy
      4. Questions
  2. Understanding Proxmox VE and Advanced Installation
    1. Understanding Proxmox features
      1. It is free!
      2. Built-in firewall
      3. Open vSwitch
      4. The graphical user interface
      5. KVM virtual machines
      6. Linux containers, or LXC
      7. Storage plugins
      8. Vibrant culture
    2. The basic installation of Proxmox
    3. The advanced installation option
    4. Debugging the Proxmox installation
    5. Proxmox subscription and repositories
      1. Proxmox VE Enterprise repository
        1. Type
        2. Subscription key
        3. Status
        4. Server ID
        5. Sockets
        6. Last checked
        7. Next due date
      2. Proxmox VE No-Subscription repository
      3. Proxmox VE Test repository
    6. Summary
  3. Creating a Cluster and Exploring the Proxmox GUI
    1. Creating a Proxmox cluster
    2. Exploring the Proxmox GUI
      1. The GUI menu system
      2. Cluster tree view
      3. Server View
      4. Folder View
      5. Storage View
      6. Pool View
      7. The Datacenter menu
        1. Datacenter | Search
        2. Datacenter | Summary
        3. Datacenter | Options
        4. Datacenter | Storage
        5. Datacenter | Backup
        6. Datacenter | Permissions
          1. Datacenter | Permissions | Users
          2. Datacenter | Permissions | Groups
          3. Datacenter | Permissions | Pools
          4. Datacenter | Permissions | Roles
          5. Datacenter | Permissions | Authentication
        7. Datacenter | HA
        8. Datacenter | Firewall
        9. Datacenter | Support
      8. Node-specific menus
        1. Node | Search
        2. Node | Summary
        3. Node | Shell
        4. Node | System
          1. Node | Network
          2. Node | DNS
          3. Node | Time
          4. Node | Syslog
        5. Node | Updates
        6. Node | Firewall
        7. Node | Disks
        8. Node | Ceph
        9. Node | Task History
        10. Node | Subscription
      9. KVM menu
        1. KVM VM | Summary
        2. KVM | Console
        3. KVM | Hardware
        4. KVM | Options
        5. KVM VM | Task History
        6. KVM | Monitor
        7. KVM | Backup
        8. KVM VM | Snapshot
        9. KVM | Firewall
        10. KVM | Permissions
      10. LXC container menu
        1. LXC container | Summary
        2. LXC container | Resources
        3. LXC container | Network
        4. LXC container | DNS
        5. LXC container | Options
        6. LXC container | Task History
        7. LXC container | Backup
        8. LXC container | Snapshots
        9. LXC container | Firewall
        10. LXC container | Permissions
      11. Pool menu
        1. Pool | Summary
        2. Pool | Members
        3. Pool | Permissions
    3. Summary
  4. Proxmox under the Hood
    1. The Proxmox cluster file system
    2. Proxmox directory structure
    3. Dissecting the configuration files
      1. The cluster configuration file
        1. logging {  }
        2. nodelist {  }
        3. quorum {  }
        4. totem {  }
        5. interface {  }
      2. Storage configuration file
      3. User configuration files
      4. The password configuration file
      5. KVM virtual machine configuration file
      6. Arguments in the KVM configuration file
      7. LXC container configuration file
      8. Version configuration file
      9. Member nodes
      10. Virtual machine list file
      11. The cluster log file
      12. Ceph configuration files
      13. Firewall configuration file
    4. Summary
  5. Storage Systems
    1. Local storage versus shared storage
      1. Live migration of a virtual machine
      2. Seamless expansion of multinode storage space
      3. Centralized backup
      4. Multilevel data tiering
      5. Central storage management
    2. Local and shared storage comparison
    3. A virtual disk image
      1. Supported image formats
        1. The .qcow2 images
        2. The .raw image type
        3. The .vmdk image type
      2. Virtual device types
      3. Managing disk images
        1. Resizing a virtual disk image
        2. Moving a virtual disk image
        3. Throttling a virtual disk image
        4. Caching a virtual disk image
      4. VirtIO bus type for Windows VMs
        1. Installing VirtIO drivers during Windows installation
        2. Installing VirtIO drivers after Windows installation
    4. Storage types in Proxmox
      1. Directory
      2. iSCSI
      3. Logical Volume Management
      4. NFS
      5. ZFS
      6. Ceph RBD
      7. GlusterFS
    5. Noncommercial/commercial storage options
    6. Summary
  6. Installing and Configuring Ceph
    1. Ceph components
      1. A physical node as cluster member
      2. Maps
        1. A cluster map
        2. A CRUSH map
      3. Monitor
      4. OSD
        1. OSD journal
      5. Metadata server
      6. PG
      7. Pools
      8. Ceph components summary
    2. Virtual Ceph for training
    3. Installing a Ceph cluster
      1. Installing Ceph on Proxmox
        1. Preparing a Proxmox node for Ceph
        2. Installing Ceph
        3. Creating mons from the Proxmox GUI
        4. Creating OSDs from Proxmox GUI
        5. Managing a Ceph pool using Proxmox GUI
        6. Creating a Ceph pool using Proxmox GUI
        7. Connecting Ceph to Proxmox
        8. Ceph command list
    4. Summary
  7. KVM Virtual Machines
    1. Exploring KVM
    2. Creating a KVM
      1. Creating a KVM using an ISO image
        1. General tab
          1. Node
          2. VM ID
          3. Name
          4. Resource Pool
          5. Help
        2. The OS tab
        3. The CD/DVD tab
        4. The Hard Disk tab
          1. Bus/Device
          2. Storage
          3. Disk size (GB)
          4. Format
          5. Cache
          6. No backup
          7. Discard
          8. IO thread
        5. The CPU tab
          1. Sockets
          2. Cores
          3. Enabling NUMA
          4. Type
        6. The Memory tab
        7. The Network tab
          1. Bridged mode
          2. Firewall
          3. NAT mode
          4. No network device
          5. Model
          6. MAC address
          7. Rate limit (MB/s)
          8. Multiqueues
          9. Disconnect
      2. Creating VM by cloning
      3. Creating VMs from a template
        1. Target node
        2. Mode
    3. Advanced configuration options for VMs
      1. Configuring a sound device
      2. Configuring PCI passthrough
      3. Configuring GPU passthrough
      4. Preparing for hotplug
      5. Configuring VMs with hotplug
        1. Hotplugging vCPUs
        2. Hotplugging memory
        3. Hotplugging disks/vNICs
    4. Migrating KVM virtual machines
    5. Summary
  8. LXC Virtual Machines
    1. Exploring LXC virtual machines
    2. Understanding container templates
    3. Creating an LXC container
      1. General tab
        1. Node
        2. CT ID
        3. Hostname
        4. Unprivileged container
        5. Resource Pool
      2. The Template tab
      3. The Root Disk tab
        1. Storage
        2. ACLs
        3. Enable quota
      4. The CPU tab
        1. Cores
      5. The Memory tab
      6. The Network tab
        1. Name
        2. MAC address
        3. Bridge
        4. The VLAN Tag
        5. Rate limit
        6. Firewall
        7. IPv4/IPv6
      7. The DNS tab
      8. The Confirm tab
    4. Managing an LXC container
      1. Adjusting resources using the GUI
      2. Adjusting resources using the CLI
      3. Adjusting resources using direct modification
    5. Migrating an LXC container
    6. Accessing an LXC container
      1. The noVNC console
      2. Direct shell through the CLI
    7. Converting OpenVZ to LXC
    8. Summary
  9. Network of Virtual Networks
    1. Exploring virtual networks
    2. Physical networks versus virtual networks
      1. A physical network
      2. A virtual network
    3. Networking components in Proxmox
      1. Virtual Network Interface Cards
        1. Adding/removing vNIC
      2. A virtual bridge
        1. Adding a virtual bridge through the GUI
          1. Name
          2. IP information
          3. Bridge ports
          4. VLAN-aware
        2. Adding a virtual bridge through CLI
      3. Extra bridge options
        1. bridge_stp
        2. bridge_fd
      4. Virtual LAN
        1. Adding a VLAN
      5. Network Address Translation/Translator
        1. Adding NAT/masquerading
      6. Network bonding
        1. Adding a bonding interface
          1. The layer 2 hash policy
          2. The layer 2+3 hash policy
          3. The layer 3+4 hash policy
      7. Multicast
        1. Configuring multicast on Netgear
    4. Open vSwitch
      1. Features of Open vSwitch
        1. Adding an Open vSwitch bridge
        2. Adding the Open vSwitch bond
        3. Adding Open vSwitch IntPort
        4. CLI for Open vSwitch
        5. Practicing Open vSwitch
          1. Configuration requirements
          2. Solutions
    5. Sample virtual networks
      1. Network #1 – Proxmox in its simplest form
      2. Network #2 – the multi-tenant environment
      3. Network #3 – academic institution
    6. A multi-tenant virtual environment
      1. A multi-tenant network diagram
    7. Summary
  10. The Proxmox VE Firewall
    1. Exploring the Proxmox VE firewall
      1. Components of the Proxmox firewall
        1. Zones
        2. Security groups
        3. IPSet
        4. Rules
        5. Protocols
        6. Macros
        7. The pve-firewall and pvefw-logger services
      2. Configuration files of a firewall
    2. Configuring the data center-specific firewall
      1. Configuring the Datacenter firewall through the GUI
        1. Creating the Datacenter firewall rules
        2. Creating the Datacenter IPSet
        3. Creating aliases
      2. Configuring the Datacenter firewall through the CLI
        1. [OPTIONS]
        2. [ALIASES]
        3. [IPSET <name>]
        4. [RULES]
        5. [group <name>]
    3. Configuring a host-specific firewall
      1. Creating host firewall rules
        1. Options for the host zone firewall
          1. Enable a firewall
          2. The SMURFS filter
          3. The TCP flags filter
          4. NDP
          5. nf_conntrack_max
          6. nf_conntrack_tcp_timeout_established
          7. log_level_in/out
          8. tcp_flags_log_level
          9. smurf_log_level
      2. Configuring the host firewall through the CLI
    4. Configuring a VM-specific firewall
      1. Creating VM firewall rules
      2. Creating  aliases
      3. Creating IPSets
      4. Options for a VM zone firewall
        1. Enable DHCP
        2. The MAC filter
        3. Input/output policy
      5. Configuring a VM-specific firewall through the CLI
    5. Integrating a Suricata IDS/IPS
      1. Installing/configuring Suricata
      2. Limitations of Suricata in Proxmox
    6. Summary
  11. Proxmox High Availability
    1. Understanding HA
      1. HA in Proxmox
      2. How Proxmox HA works
    2. Requirements for HA setup
      1. At least three nodes
      2. Shared storage
      3. Fencing
      4. BIOS power-on feature
    3. Configuring Proxmox HA
      1. The HA menu
        1. Status
        2. The Resources menu
      2. The Groups menu
        1. ID
        2. Node
        3. The restricted checkbox
        4. The nofailback checkbox
      3. The Fencing menu
    4. Testing Proxmox HA configuration
    5. The Proxmox HA simulator
      1. Configuring the Proxmox HA simulator
    6. Summary
  12. Monitoring the Proxmox Cluster
    1. An introduction to monitoring
    2. Proxmox built-in monitoring
      1. Datacenter Status
      2. Node Status
    3. Zabbix as a monitoring solution
    4. Installing Zabbix
    5. Configuring Zabbix
      1. Configuring a host to monitor
      2. Displaying data using a graph
      3. Configuring the disk health notification
        1. Installing smart monitor tools
        2. Configuring the Zabbix agent
        3. Creating a Zabbix item in the GUI
        4. Creating a trigger in the GUI
        5. Creating graphs in the GUI
    6. Configuring SNMP in Proxmox
      1. Object Identifiers
      2. Management Information Base
    7. Adding an SNMP device in Zabbix
    8. Monitoring the Ceph cluster with the Proxmox GUI
    9. Monitoring a Ceph cluster with third-party options
    10. Summary
  13. Proxmox Production-Level Setup
    1. Defining the production level
      1. Key components
        1. Stable and scalable hardware
        2. Redundancy
          1. Node level
          2. Utility level
          3. Network level
          4. HVAC level
          5. Storage level
        3. Current load versus future growth
        4. Budget
        5. Simplicity
        6. Tracking hardware inventory
        7. Hardware selection
    2. Sizing CPU and memory
      1. Single socket versus multi-socket
      2. Hyper-threading – enable versus disable
      3. Start small with VM resources
      4. Balancing node resources
    3. Ceph cluster production
      1. Forget about hardware RAID
      2. Solid State Drive for Ceph Journal
      3. Network bandwidth
    4. Liquid cooling
      1. Total immersion in oil
      2. Total immersion in 3M Novec
      3. Direct contact liquid cooling
    5. Real-world Proxmox scenarios
      1. Scenario 1 – an academic institution
      2. Scenario 2 – multi-tier storage cluster with a Proxmox cluster
      3. Scenario 3 - Virtual infrastructure for a multi-tenant cloud service provider
      4. Scenario 4 – nested virtual environment for a software development company
      5. Scenario 5 – virtual infrastructure for a public library
      6. Scenario 6 – multi-floor office virtual infrastructure with virtual desktops
      7. Scenario 7 – virtual infrastructure for the hotel industry
      8. Scenario 8 – virtual infrastructure for geological survey organization
    6. Summary
  14. Back Up and Restore Virtual Machines
    1. Proxmox backup options
      1. A full backup
        1. Full backup modes
          1. Snapshot
          2. Suspend
          3. Stop
        2. Backup compression
          1. None
          2. LZO
          3. GZIP
      2. Snapshots
    2. Configuring backup storage
      1. Show VM configuration from backup
    3. Configuring full backup
      1. Creating a schedule for backup
        1. Node
        2. Storage
        3. Day of week
        4. Start Time
        5. Selection mode
        6. Send email to
        7. Email notification
        8. Compression
        9. Mode
        10. Enable
      2. Creating a manual backup
    4. Creating snapshots
    5. Restoring a virtual machine
    6. Backup/restore through the CLI
      1. Backup using the CLI
      2. Restore using the CLI
      3. Unlocking a VM after a backup error
    7. Virtual machine replication
      1. Creating a replication task through the GUI
        1. Target
        2. Schedule
        3. Rate limit (MB/s)
        4. Enabled
      2. Creating a replication task through the CLI
      3. Replication process
    8. Backup configuration file
      1. The bwlimit option
      2. The lockwait option
      3. The stopwait option
      4. The stdexcludes option
      5. The mailto option
      6. The script option
      7. The exclude-path option
      8. The pigz option
    9. Summary
  15. Updating/Upgrading Proxmox
    1. Introducing Proxmox updates
    2. Updating Proxmox through the GUI
    3. Updating Proxmox through the CLI
      1. Difference between upgrade and dist-upgrade
    4. Recovering from the grub2 update issue
    5. Updating after a subscription change
    6. Rebooting dilemma after Proxmox updates
      1. Applying update without reboot
    7. Summary
  16. Proxmox Troubleshooting
    1. Proxmox node issues
      1. Issue – fresh Proxmox install stuck with /dev to be a fully populated error during node reboot
      2. Issue – rejoining a node to a Proxmox node with the same old IP address
      3. Issue – Proxmox installation completed but grub is in an endless loop after reboot
      4. Issue – LSI MegaRAID 9240-8i/9240-4i causes an error during booting of the Proxmox node
        1. Downloading and updating the LSI driver
        2. Updating the Supermicro BIOS
      5. Issue – the Upgrade button is disabled on the Proxmox GUI, which prevents the node upgrade
      6. Issue – Proxmox cannot start due to the getpwnam error
      7. Issue – cannot log in to the GUI as root after reinstalling Proxmox on the same node
    2. The main cluster issues
      1. Issue – Proxmox virtual machines are running, but the Proxmox GUI shows that everything is offline
      2. Issue – kernel panic when disconnecting USB devices, such as a keyboard, mouse, or UPS
      3. Issue – virtual machines on Proxmox will not shut down if shutdown is initiated from the Proxmox GUI
      4. Issue – kernel panic with HP NC360T (Intel 82571EB chipset) only in Proxmox VE 3.2
      5. Issue – the Proxmox cluster is out of quorum and cluster filesystem is in read-only mode
      6. Issue – VM will not respond to shutdown or restart
      7. Issue – Proxmox GUI not responding after Firefox update
      8. Issue – the Proxmox GUI is not showing RRD graphs
    3. Storage issues
      1. Issue – deleting a damaged LVM from Proxmox with the error read failed from 0 to 4096
      2. Issue – Proxmox cannot mount NFS share due to the timing out error
      3. Issue – how to delete leftover NFS shares in Proxmox or what to do when the NFS stale file handle error occurs?
      4. Issue – Proxmox issues --mode session exit code 21 errors while trying to access the iSCSI target
      5. Issue – cannot read an iSCSI target even after it has been deleted from Proxmox storage
      6. Issue – a Ceph node is removed from the Proxmox cluster, but OSDs still show up in PVE
      7. Issue – the no such block device error during creation of an OSD through the Proxmox GUI
      8. Issue – the fstrim command does not trim unused blocks for the Ceph storage
      9. Issue – the RBD couldn't connect to cluster (500) error when connecting Ceph with Proxmox
      10. Issue – changing the storage type from IDE to VirtIO after the VM has been set up and the OS has been installed
      11. Issue – the pveceph configuration not initialized (500) error when you click on the Ceph tab in the Proxmox GUI
      12. Issue – the CephFS storage disappears after a Proxmox node reboots
      13. Issue – VM cloning does not parse in the Ceph storage
      14. Issue – VM disk images stored on ZFS is extremely slow
    4. Network connectivity issues
      1. Issue – no connectivity on Realtek RTL8111/8411 rev. 06 network interfaces
      2. Issue – network performance is slower with the E1000 virtual network interfaces
      3. Issue – patch port for Open vSwitch in Proxmox not working
      4. Issue – trying to add a node to a newly created Proxmox cluster when nodes do not form quorum
      5. Issue – implemented IPv6 but firewall rules do not get applied
    5. KVM virtual machine issues
      1. Issue – Windows 7/XP machine converted to Proxmox KVM hangs during boot
      2. Issue – Windows 7 VM does not reboot, instead it shuts down, requiring a manual boot from Proxmox
      3. Issue – the qemu-img command does not convert the .vmdk image files created with the .ova template in Proxmox VE 5.0
      4. Issue – online migration of a virtual machine fails with a failed to sync data error
      5. Issue – no audio in Windows KVM
      6. Issue – the VirtIO virtual disk is not available during the Windows Server installation
    6. LXC container issues
      1. Issue – a Proxmox node hangs when trying to stop or restart an LXC container
      2. Issue – the noVNC console only shows a cursor for LXC containers
    7. Backup/restore issues
      1. Issue – a Proxmox VM is locked after backup crashes unexpectedly
      2. Issue – how can Proxmox back up only the primary OS virtual disk instead of all the virtual disks for a VM?
      3. Issue – backup of virtual machines stops prematurely with an operation not permitted error
      4. Issue – a backup task takes a very long time to complete, or it crashes when multiple nodes are backing up to the same backup storage
      5. Issue – backup of virtual machines aborts a backup task prematurely
      6. Issue – backup storage has a lot of .dat files and .tmp folders using the storage space
    8. VNC/SPICE console issues
      1. Issue – the mouse pointer is not shared with SPICE (virt-viewer) on Windows 8 VM
      2. Issue – remote viewer is unable to connect to a SPICE-enabled virtual machine on the Windows OS
    9. Firewall issues
      1. Issue – rules are created and a firewall is enabled for vNIC, but rules do not get applied
      2. Issue – a firewall is enabled for a VM and the necessary rules are created, but nothing is being filtered for that VM
    10. Summary
  17. Rescuing Proxmox
    1. Recovering from OS drive failure
      1. Physical drive failure
      2. OS data corruption
      3. Migrating VMs from a faulty node
      4. Reinstalling Proxmox
    2. Recovering from a quorum failure
    3. Recovering from a node failure
    4. Recovering from a network failure
      1. Loss of connectivity between Proxmox nodes
      2. Loss of connectivity between Proxmox nodes and users
      3. Loss of connectivity between Proxmox and storage nodes
    5. Recovering from Ceph failure
      1. Best practices for a healthy Ceph cluster
      2. Stuck inconsistent PGs in Ceph
      3. Stuck inactive incomplete PGs in Ceph
      4. Error while moving a Ceph journal to another drive
      5. Ceph node running out of resources during recovery
    6. Summary

Product information

  • Title: Mastering Proxmox - Third Edition
  • Author(s): Wasim Ahmed
  • Release date: November 2017
  • Publisher(s): Packt Publishing
  • ISBN: 9781788397605