April 2016
Intermediate to advanced
170 pages
3h 48m
English
Content preview from Distributed Computing with PythonBecome an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Storing data in Amazon S3
Amazon Simple Storage Service, S3, is a web service that can be used to store and retrieve arbitrary blobs of data. Data stored in S3 can include files of any kind, up to 5 terabytes in size (at the time of writing this), and also raw bytes.
S3 is also significantly cheaper than EBS; however, it does not offer a filesystem layer but rather a REST API. Another difference is that while EBS volumes can only be attached to a single running instance at a time, S3 objects can be shared among as many instances as we want, and depending on the desired permission policy, they can be accessed from anywhere on the Internet.
Getting started with S3 is easy; you need to create a number of buckets (that is, data containers in S3 parlance) ...