Mastering KVM Virtualization - Second Edition

Mastering KVM Virtualization - Second Edition

by Vedran Dakic, Humble Devassy Chirammal, Prasad Mukhedkar, Anil Vettathu
Released October 2020
Publisher(s): Packt Publishing
ISBN: 9781838828714

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Learn how to configure, automate, orchestrate, troubleshoot, and monitor KVM-based environments capable of scaling to private and hybrid cloud models

Key Features

  • Gain expert insights into Linux virtualization and the KVM ecosystem with this comprehensive guide
  • Learn to use various Linux tools such as QEMU, oVirt, libvirt, Cloud-Init, and Cloudbase-Init
  • Scale, monitor, and troubleshoot your VMs on various platforms, including OpenStack and AWS

Book Description

Kernel-based Virtual Machine (KVM) enables you to virtualize your data center by transforming your Linux operating system into a powerful hypervisor that allows you to manage multiple operating systems with minimal fuss. With this book, you'll gain insights into configuring, troubleshooting, and fixing bugs in KVM virtualization and related software.

This second edition of Mastering KVM Virtualization is updated to cover the latest developments in the core KVM components - libvirt and QEMU. Starting with the basics of Linux virtualization, you'll explore VM lifecycle management and migration techniques. You'll then learn how to use SPICE and VNC protocols while creating VMs and discover best practices for using snapshots. As you progress, you'll integrate third-party tools with Ansible for automation and orchestration. You'll also learn to scale out and monitor your environments, and will cover oVirt, OpenStack, Eucalyptus, AWS, and ELK stack. Throughout the book, you'll find out more about tools such as Cloud-Init and Cloudbase-Init. Finally, you'll be taken through the performance tuning and troubleshooting guidelines for KVM-based virtual machines and a hypervisor.

By the end of this book, you'll be well-versed with KVM virtualization and the tools and technologies needed to build and manage diverse virtualization environments.

What you will learn

  • Implement KVM virtualization using libvirt and oVirt
  • Delve into KVM storage and network
  • Understand snapshots, templates, and live migration features
  • Get to grips with managing, scaling, and optimizing the KVM ecosystem
  • Discover how to tune and optimize KVM virtualization hosts
  • Adopt best practices for KVM platform troubleshooting

Who this book is for

If you are a systems administrator, DevOps practitioner, or developer with Linux experience looking to sharpen your open-source virtualization skills, this virtualization book is for you. Prior understanding of the Linux command line and virtualization is required before getting started with this book.

Table of contents

  1. Mastering KVM Virtualization Second Edition
  2. Why subscribe?
  3. Contributors
  4. About the authors
  5. About the reviewer
  6. Packt is searching for authors like you
  7. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Code in Action
    5. Download the color images
    6. Conventions used
    7. Get in touch
    8. Reviews
  8. Section 1: KVM Virtualization Basics
  9. Chapter 1: Understanding Linux Virtualization
    1. Linux virtualization and how it all started
    2. Types of virtualization
    3. Using the hypervisor/virtual machine manager
      1. Type 1 and type 2 hypervisors
    4. Open source virtualization projects
      1. Xen
      2. KVM
    5. What Linux virtualization offers you in the cloud
    6. Summary
    7. Questions
    8. Further reading
  10. Chapter 2: KVM as a Virtualization Solution
    1. Virtualization as a concept
      1. Virtualized versus physical environments
      2. Why is virtualization so important?
      3. Hardware requirements for virtualization
      4. Software requirements for virtualization
    2. The internal workings of libvirt, QEMU, and KVM
      1. libvirt
      2. QEMU
      3. QEMU – KVM internals
      4. Data structures
      5. Threading models in QEMU
      6. KVM
      7. Data structures
    3. Execution flow of vCPU
    4. Summary
    5. Questions
    6. Further reading
  11. Section 2: libvirt and ovirt for Virtual Machine Management
  12. Chapter 3: Installing KVM Hypervisor, libvirt, and oVirt
    1. Getting acquainted with QEMU and libvirt
    2. Getting acquainted with oVirt
    3. Installing QEMU, libvirt, and oVirt
      1. Installing the first virtual machine in KVM
      2. Automating virtual machine installation
      3. Installing oVirt
    4. Starting a virtual machine using QEMU and libvirt
    5. Summary
    6. Questions
    7. Further reading
  13. Chapter 4: Libvirt Networking
    1. Understanding physical and virtual networking
    2. Virtual networking
      1. Libvirt NAT network
      2. Libvirt routed network
      3. Libvirt isolated network
    3. Using userspace networking with TAP and TUN devices
    4. Implementing Linux bridging
    5. Configuring Open vSwitch
      1. Other Open vSwitch use cases
    6. Understanding and using SR-IOV
    7. Understanding macvtap
    8. Summary
    9. Questions
    10. Further reading
  14. Chapter 5: Libvirt Storage
    1. Introduction to storage
    2. Storage pools
      1. Local storage pools
      2. Libvirt storage pools
    3. NFS storage pool
    4. iSCSI and SAN storage
    5. Storage redundancy and multipathing
    6. Gluster and Ceph as a storage backend for KVM
      1. Gluster
      2. Ceph
    7. Virtual disk images and formats and basic KVM storage operations
      1. Getting image information
      2. Attaching a disk using virt-manager
      3. Attaching a disk using virsh
      4. Creating an ISO image library
      5. Deleting a storage pool
      6. Creating storage volumes
      7. Creating volumes using the virsh command
      8. Deleting a volume using the virsh command
    8. The latest developments in storage – NVMe and NVMeOF
    9. Summary
    10. Questions
    11. Further reading
  15. Chapter 6: Virtual Display Devices and Protocols
    1. Using virtual machine display devices
      1. Physical and virtual graphics cards in VDI scenarios
      2. GPU PCI passthrough
    2. Discussing remote display protocols
      1. Remote display protocols history
      2. Types of remote display protocols
    3. Using the VNC display protocol
      1. Why VNC?
    4. Using the SPICE display protocol
      1. Adding a SPICE graphics server
    5. Methods to access a virtual machine console
    6. Getting display portability with noVNC
    7. Summary
    8. Questions
    9. Further reading
  16. Chapter 7: Virtual Machines: Installation, Configuration, and Life Cycle Management
    1. Creating a new VM using virt-manager
      1. Using virt-manager
      2. Using virt-* commands
      3. Creating a new VM using Cockpit
    2. Creating a new VM using oVirt
    3. Configuring your VM
    4. Adding and removing virtual hardware from your VM
    5. Migrating VMs
      1. Benefits of VM migration
      2. Setting up the environment
      3. Offline migration
      4. Live or online migration
    6. Summary
    7. Questions
    8. Further reading
  17. Chapter 8: Creating and Modifying VM Disks, Templates, and Snapshots
    1. Modifying VM images using libguestfs tools
      1. virt-v2v
      2. virt-p2v
      3. guestfish
    2. VM templating
      1. Working with templates
      2. Deploying VMs from a template
    3. virt-builder and virt-builder repos
      1. virt-builder repositories
    4. Snapshots
      1. Working with internal snapshots
      2. Managing snapshots using virt-manager
      3. Working with external disk snapshots
    5. Use cases and best practices while using snapshots
    6. Summary
    7. Questions
    8. Further reading
  18. Section 3: Automation, Customization, and Orchestration for KVM VMs
  19. Chapter 9: Customizing a Virtual Machine with cloud-init
    1. What is the need for virtual machine customization?
    2. Understanding cloud-init
    3. Understanding cloud-init architecture
    4. Installing and configuring cloud-init at boot time
      1. Cloud-init images
      2. Cloud-init data sources
    5. Passing metadata and user data to cloud-init
      1. Using cloud-init modules
    6. Examples on how to use a cloud-config script with cloud-init
      1. The first deployment
      2. The second deployment
      3. The third deployment
    7. Summary
    8. Questions
    9. Further reading
  20. Chapter 10: Automated Windows Guest Deployment and Customization
    1. The prerequisites to creating Windows VMs on KVM
    2. Creating Windows VMs using the virt-install utility
    3. Customizing Windows VMs using cloudbase-init
    4. cloudbase-init customization examples
    5. Troubleshooting common cloudbase-init customization issues
    6. Summary
    7. Questions
    8. Further reading
  21. Chapter 11: Ansible and Scripting for Orchestration and Automation
    1. Understanding Ansible
      1. Automation approaches
      2. Introduction to Ansible
      3. Deploying and using AWX
      4. Deploying Ansible
    2. Provisioning a virtual machine using the kvm_libvirt module
      1. Working with playbooks
      2. Installing KVM
      3. Using Ansible and cloud-init for automation and orchestration
    3. Orchestrating multi-tier application deployment on KVM VM
    4. Learning by example – various examples of using Ansible with KVM
    5. Summary
    6. Questions
    7. Further reading
  22. Section 4: Scalability, Monitoring, Performance Tuning, and Troubleshooting
  23. Chapter 12: Scaling Out KVM with OpenStack
    1. Introduction to OpenStack
    2. Software-defined networking
      1. Understanding VXLAN
      2. Understanding GENEVE
    3. OpenStack components
      1. Swift
      2. Nova
      3. Glance
      4. Horizon
      5. Designate
      6. Keystone
      7. Neutron
    4. Additional OpenStack use cases
      1. Creating a Packstack demo environment for OpenStack
    5. Provisioning the OpenStack environment
      1. Installing OpenStack step by step
      2. OpenStack administration
      3. Day-to-day administration
      4. Identity management
    6. Integrating OpenStack with Ansible
      1. Installing an Ansible deployment server
      2. Configuring the Ansible inventory
      3. Running Ansible playbooks
    7. Summary
    8. Questions
    9. Further reading
  24. Chapter 13: Scaling out KVM with AWS
    1. Introduction to AWS
      1. Approaching the cloud
      2. Multi-cloud
      3. Shadow IT
      4. Market share
      5. Big infrastructure but no services
      6. Pricing
      7. Data centers
      8. Placement is the key
      9. AWS services
    2. Preparing and converting virtual machines for AWS
      1. What do we want to do?
      2. Uploading an image to EC2
    3. Building hybrid KVM clouds with Eucalyptus
      1. How do you install it?
      2. Using Eucalyptus for AWS control
    4. Summary
    5. Questions
    6. Further reading
  25. Chapter 14: Monitoring the KVM Virtualization Platform
    1. Monitoring the KVM virtualization platform
    2. Introduction to the open source ELK solution
      1. Elasticsearch
      2. Logstash
      3. Kibana
    3. Setting up and integrating the ELK stack
      1. Workflow
    4. Configuring data collector and aggregator
      1. Creating charts in Kibana
      2. Creating custom utilization reports
      3. ELK and KVM
    5. Summary
    6. Questions
    7. Further reading
  26. Chapter 15: Performance Tuning and Optimization for KVM VMs
    1. It's all about design
      1. General hardware design
      2. VM design
    2. Tuning the VM CPU and memory performance
      1. CPU pinning
      2. Working with memory
    3. Getting acquainted with KSM
    4. Tuning the CPU and memory with NUMA
      1. NUMA memory allocation policies
      2. Understanding emulatorpin
      3. KSM and NUMA
      4. Automatic NUMA balancing
      5. The numactl command
      6. Understanding numad and numastat
    5. Virtio device tuning
    6. Block I/O tuning
    7. Network I/O tuning
      1. How to turn it on
      2. KVM guest time-keeping best practices
      3. Software-based design
    8. Summary
    9. Questions
    10. Further reading
  27. Chapter 16: Troubleshooting Guidelines for the KVM Platform
    1. Verifying the KVM service status
    2. KVM services logging
    3. Enabling debug mode logging
    4. Advanced troubleshooting tools
      1. oVirt
      2. oVirt and KVM storage problems
      3. Problems with snapshots and templates – virtual machine customization
      4. Problems working with Ansible and OpenStack
      5. Dependencies
      6. Troubleshooting Eucalyptus
      7. AWS and its verbosity, which doesn't help
      8. Paying attention to details
      9. Troubleshooting problems with the ELK stack
    5. Best practices for troubleshooting KVM issues
    6. Summary
    7. Questions
    8. Further reading
  28. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Mastering KVM Virtualization - Second Edition
  • Author(s): Vedran Dakic, Humble Devassy Chirammal, Prasad Mukhedkar, Anil Vettathu
  • Release date: October 2020
  • Publisher(s): Packt Publishing
  • ISBN: 9781838828714