Programming Amazon Web Services

S3 Filesystem with ElasticDrive

One of the most interesting potential uses for S3 is as an unlimited data store on top of which other filesystem interface abstractions can be built. Appendix A lists a number of products or services that use S3 as an underlying storage repository but expose it via different file or storage management protocols.

Some of these tools are designed to make S3 storage resources accessible to existing network-based tools that do not recognize S3—for example, as a File Transfer Protocol (FTP) or a Web-based Distributed Authoring and Versioning (WebDAV) service—and others aim to make the storage space in S3 available as a lower-level filesystem resource. Both of these approaches provide benefits, but it is the S3-based filesystem approach that we will concentrate on in this section, because it presents the most interesting possibilities. It also presents the most difficult challenges.

If it proves to be feasible to build whole filesystems on top of S3, many of the service’s limitations could be overcome in a very elegant way. Rather than having to use specialized S3 tools to access your storage space, you can make the service look and behave like a standard disk drive that stores data reliably in the cloud behind the scenes. On your computer you could copy files to and from this disk, even rename and rearrange them. In the background the changes you make would automatically be translated into API requests and stored in S3. This approach also makes it possible ...