OpenStack Swift

A Swift Afterword

Thanks for spending some time with OpenStack Swift. We hope it’s been helpful and informative. Before wrapping up, we wish you all the best in your work with Swift, and invite your feedback.

To close, we’d like to reflect on some of our recent experiences working with Swift and try to look ahead. Much of this will echo what we’ve been saying throughout the book, but we hope that this will reassure you that by choosing Swift you’re positioning yourself—and your organization—on the right side of history (at least as far as object storage goes).

Over the past few years, we’ve had the privilege of working with hundreds of architects, operators, and deployers of private cloud-storage infrastructure. We observed three noteworthy trends: (1) a pervasive transition to object-based storage, (2) a strong desire to use open source software, and (3) the adoption of OpenStack Swift as the standard for object storage.

The Transition to Object Storage

In 2013, adoption of public object-storage services continued at an ever-accelerating rate. In April, Amazon announced that 1 trillion objects had been uploaded to S3 in the prior 10 months. For a bit of context, it took six years for the first 1 trillion objects to be uploaded. By some estimates, S3 generated several hundred millions of dollars of revenue for Amazon in 2013—and grew over 100% annually. HP, IBM, Google, Microsoft, Rackspace, and Oracle also grew or launched their public object-storage offerings.

Much of this amazing growth is fueled by the numerous web, mobile, and software-as-a-service (SaaS) applications that depend on object storage. For these applications, the performance, scale, and durability of object storage is essential. Traditional filesystem technologies cannot provide the dramatic increase of scale that these applications require. Arrays do not meet the price-per-capacity needs either. Object storage is quite simply the natural storage platform for applications being built today.

Companies looking to store vast amounts of archival data are also benefiting from the cost advantages of object storage. Even major tape vendors are now promoting object-based technologies to meet the needs of increased scale and to lower the cost of long-term archives.

Why Being Open Matters

OpenStack Swift is open source software. To understand why this matters for storage architects and operators, let’s look back at what happened to the rest of the infrastructure stack. Previously dismissed by their proprietary competitors as “immature,” open source operating systems, middleware, application frameworks, and databases are now standards in enterprise and web infrastructure. The open source model has so completely and fundamentally transformed the infrastructure tier in the data center that very few proprietary infrastructure platform technologies still have a sustainable competitive advantage.

That same transformation is now changing the storage tier, one organization at a time. Vendors of proprietary storage technology often claim that open source cannot produce enterprise-grade storage solutions. However, when done the right way (open code, flexible design, large community), open source platforms not only produce enterprise-grade storage solutions, they provide ones that are the clear winner over locked-in, failure-vulnerable systems. Today, using open source platforms is an irreversible change in enterprise architectures.

It is important to remember that using an open source storage engine does not preclude working with commercial vendors in the OpenStack ecosystem. Jonathan Bryce, executive director of the OpenStack Foundation, explained it this way at the November 2013 OpenStack Summit.

Cutting edge technology from an open source project is being used by some of the biggest companies in the world. How do they get a comfort level deploying this technology? Partners and support from the community, like SwiftStack and others in the ecosystem that have helped Enterprises meet their business needs.

The Swift ecosystem is very active with great development velocity and real diversity. Swift is much larger than what any single storage company could build. Over 166 developers have contributed to the OpenStack Swift codebase. The Havana release of Swift, which included support for global clusters, had top contributors (by patch count) from five different companies: SwiftStack, Rackspace, Red Hat, eNovance, and IBM. With the recent Icehouse release, Intel, Red Hat, and SwiftStack all worked on storage policies for Swift. It’s this ecosystem, in large part, that is driving enterprises to adopt Swift for their production object-storage needs and this ecosystem that will be the reason why Swift continues to succeed.

The Object-Storage Standard

The Swift API is increasingly becoming the API of choice. Terri McClure, senior analyst at Enterprise Strategy Group, notes, “The Swift API has been adopted by or is on the roadmap for most vendors as an object-storage interface, enabling those vendors to serve public, private, and hybrid cloud deployments without concerns about users making proprietary API investments.” Naturally, the biggest cloud operators have APIs that developers use so applications can write to those storage clouds. EMC, Oracle, and Red Hat are a few of the largest vendors who announced support for the Swift API in 2013. This allows their users to integrate with public clouds that run Swift, and leverage private clouds riding the popularity of Swift in private deployments.

We predict that the future will further cement Swift as the standard for object storage.

Now It’s Your Turn

Now it’s your turn—in two ways. First, it’s your turn to start applying and working with Swift, SwiftStack, and anything else that you’re integrating with. We hope that this process will be satisfying and fulfilling. There’s a great community of Swift operators and developers you can turn to for help. And we’re always happy to hear from you ([e-mail]).

Second, it’s your turn—if you wish—to share with us your comments, feedback, and suggestions. We’re planning to produce a second edition of this book and we’re very interested in your sense of what we need to correct or add. Thanks, in advance, for helping us create a resource that will serve you better. You can share comments and feedback by visiting this book’s website.

Thanks and we look forward to hearing from you, working with you, or just seeing you around.