OpenStack Object Storage (Swift) Essentials

OpenStack Object Storage (Swift) Essentials

by Amar Kapadia, Kris Rajana, Sreedhar Varma
Released May 2015
Publisher(s): Packt Publishing
ISBN: 9781785283598

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Design, implement, and successfully manage your cloud storage using OpenStack Swift

In Detail

OpenStack is a rapidly growing open source cloud software; and Swift is one such project. Swift allows users to build cloud storage with the help of inexpensive commodity hardware. With the latest OpenStack Juno release, developers and experts have access to hundreds of new features to build robust cloud infrastructures.

This book teaches you the fundamentals of cloud storage and OpenStack Swift and gives you the skillset to build and operate your own cloud storage. This book also explains the usefulness of OpenStack Swift for cloud storage and how it provides an architectural review of Swift. Next, the book covers the installation and management of Swift, along with other new topics including quality of service and storage policies. Hardware choice, Swift tuning, and use cases will round out your skills. This edition also features two new chapters covering OpenvStorage and Sahara. The book then demonstrates monitoring and new packaging methods, including Docker. Get a head start in the world of cloud storage using this content-rich book loaded with topics.

What You Will Learn

  • Learn the intricacies of cloud storage and discover the new features of Swift
  • Understand OpenStack Swift's architecture and install a multicluster environment
  • Use new packaging techniques such as Docker
  • Perform basic and advanced operations for day-to-day management using REST APIs
  • Know how to choose the right hardware configuration for your needs
  • Tune Swift for your particular workload and use case
  • Explore new Swift interfaces such as Cinder and Sahara

Table of contents

  1. OpenStack Object Storage (Swift) Essentials
    1. Table of Contents
    2. OpenStack Object Storage (Swift) Essentials
    3. Credits
    4. About the Authors
    5. About the Reviewers
    6. www.PacktPub.com
      1. Support files, eBooks, discount offers, and more
        1. Why subscribe?
        2. Free access for Packt account holders
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Errata
        2. Piracy
        3. Questions
    8. 1. Cloud Storage – Why Can't I Be Like Google?
      1. What constitutes cloud storage?
        1. Reduced TCO
        2. Unlimited scalability
        3. Elastic
        4. On-demand
        5. Universal access
        6. Multitenancy
        7. Data durability and availability
      2. Limitations of cloud storage
        1. Performance
        2. New APIs
      3. Object storage
      4. The importance of being open
      5. OpenStack Swift
      6. Summary
    9. 2. OpenStack Swift Architecture
      1. Logical organization of objects
      2. Swift implementation and architecture
        1. Key architectural principles
        2. Physical data organization
        3. Data path software servers
          1. A day in the life of a create operation
          2. A day in the life of a read operation
          3. A day in the life of an update operation
          4. A day in the life of a delete operation
        4. Post-processing software components
          1. Replication
          2. Updaters
          3. Auditors
          4. Other processes
        5. Inline middleware options
          1. Authentication
          2. Other modules
      3. Additional features
        1. Large object support
        2. Metadata
        3. Multirange support
        4. CORS
        5. Server-side copies
        6. Cluster health
      4. Summary
    10. 3. Installing OpenStack Swift
      1. Hardware planning
      2. Server setup and network configuration
        1. Pre-installation steps
        2. Downloading and installing Swift
        3. Setting up storage server nodes
          1. Installing services
          2. Formatting and mounting hard disks
          3. RSYNC and RSYNCD
        4. Setting up the proxy server node
        5. The Keystone service
          1. Installing MariaDB
          2. Installing Keystone
        6. The ring setup
        7. Multiregion support
      3. Finalizing the installation
      4. Storage policies
        1. Implementing storage policies
        2. Applying storage policies
      5. Summary
    11. 4. Using Swift
      1. Installing clients
      2. Creating a token using Keystone authentication
      3. Displaying metadata information for an account, container, or object
        1. Using the Swift client CLI
        2. Using cURL
        3. Using the specialized REST API client
      4. Listing containers
        1. Using the Swift client CLI
        2. Using cURL
      5. Listing objects in a container
        1. Using the Swift client CLI
        2. Using cURL
        3. Using the REST API
      6. Updating the metadata for a container
        1. Using the Swift client CLI
        2. Using the REST API
      7. Environment variables
      8. The pseudo-hierarchical directories
      9. Container ACLs
      10. Transferring large objects
      11. Amazon S3 API compatibility
        1. Accessing Swift using S3 commands
      12. Accessing Swift using client libraries
        1. Java
        2. Python
        3. Ruby
      13. Summary
    12. 5. Additional Swift Interfaces
      1. Using Swift for virtual machine storage
      2. Swift in Sahara
        1. Hadoop Cluster with Sahara
        2. Using Swift with Sahara
        3. Running a job in Sahara
        4. Authenticating with Swift proxy
      3. Summary
    13. 6. Monitoring and Managing Swift
      1. Routine management
        1. Swift cluster monitoring
          1. Swift Recon
          2. Swift Informant
          3. Swift dispersion tool
          4. StatsD
          5. Swift metrics
      2. Tulsi – a Swift health monitoring tool
      3. Architecture of Tulsi
        1. Deploying Tulsi
        2. Running Tulsi
        3. Anomaly detection in Tulsi
      4. Logging using rsyslog
      5. Failure management
        1. Detecting drive failures
        2. Handling drive failure
        3. Handling node failure
          1. Proxy server failure
        4. Zone and region failure
      6. Capacity planning
        1. Adding new drives
        2. Adding new storage and proxy servers
      7. Migrations
      8. Summary
    14. 7. Docker Intercepts Swift
      1. Swift with Docker
      2. Installation of Docker
      3. Basic commands for the Docker user
        1. Setting up a Swift proxy container using the Docker image
        2. Setting up the storage container using the Docker image
      4. Setting up a Swift cluster using a Dockerfile
        1. Creating a proxy container using a Dockerfile
        2. Creating a storage container using a Dockerfile
      5. Summary
    15. 8. Choosing the Right Hardware
      1. The hardware list
      2. The hardware selection criteria
        1. Choosing the storage server configuration
        2. Determining the region and zone configuration
        3. Choosing the account and container server configuration
        4. Selecting the proxy server configuration
        5. Choosing the network hardware
        6. Choosing the ratios of various server types
        7. Heterogeneous hardware
        8. Choosing additional networking equipment
        9. Selecting a cloud gateway
      3. Additional selection criteria
      4. The vendor selection strategy
        1. Branded hardware
        2. Commodity hardware
      5. Summary
    16. 9. Tuning Your Swift Installation
      1. Performance benchmarking
      2. Hardware tuning
      3. Software tuning
        1. Ring considerations
        2. Data path software tuning
        3. Post-processing software tuning
      4. Additional tuning parameters
      5. Summary
    17. 10. Additional Resources
      1. Use cases
        1. Archival
        2. Backup
        3. Content repository
        4. Collaboration
        5. Data lakes
      2. Operating systems used for OpenStack implementations
      3. Virtualization used for OpenStack implementations
      4. Provisioning and distribution tools
      5. Monitoring and graphing tools
      6. Additional information
      7. Summary
    18. A. Swift CLI Commands
      1. Commands
        1. list
          1. Examples
        2. stat
          1. Examples
        3. post
          1. Examples
        4. upload
          1. Examples
        5. download
          1. Examples
        6. delete
          1. Examples
    19. Index

Product information

  • Title: OpenStack Object Storage (Swift) Essentials
  • Author(s): Amar Kapadia, Kris Rajana, Sreedhar Varma
  • Release date: May 2015
  • Publisher(s): Packt Publishing
  • ISBN: 9781785283598