Programming Amazon Web Services

Chapter 4. S3 Applications

The Amazon Simple Storage Service (S3) can be used in a number of ways to meet many different storage needs. You can use it as a basic online file repository for backing up files, for web site hosting, as the basis for a network-mounted filesystem, or as a distribution network. In this chapter we discuss how you can use S3 to fulfill some common tasks by taking advantage of some of the available tools, software libraries, or third-party services.

The S3 developer ecosystem is very active, and much of the third-party software available is open source and free. In many cases you can achieve a great deal without having to write your own code, and even if you have specific needs not yet met by existing software, there are mature libraries available in a range of languages that you can use to build your own solution.

Share Large Files

A very simple application for S3 involves using the service as a repository for sharing files that are too large to include in an email. There are a number of online services already available to do this job, but many charge monthly subscription fees if you need to share very large files; with S3 you can do this yourself at little cost.

To share a file with your friends or colleagues, you will need to upload the file to S3 and send a URI link to the S3 object in an email. Because your files may contain private information, we will keep them private by generating a signed Universal Resource Identifier (URI) link to the object so that only the people who receive the link from you can access it. An advantage of using a signed URI is that you can choose how long the link will remain valid.

Example 4-1 defines a simple Ruby script that will upload a file to S3 and print out a signed URI you can share with others. Save this script as sharefile.rb, then modify the BUCKET_NAME constant to reference a bucket you have already created in your S3 account.

Example 4-1. Share file script: sharefile.rb

# The name of the bucket where shared files will be stored
BUCKET_NAME = 'my-bucket'

require 'S3'

if __FILE__ == $0
  if ARGV.length < 1
      puts "Usage: #{$0} file_to_upload [hours_until_expiry]"
      puts "Links will be valid for 24 hours by default"
      exit
  end

  # Calculate the expiry time in seconds.
  hours_until_expiry = (ARGV[1].nil? ? 24 : ARGV[1].to_f)
  expiry_time = Time.now.to_i + (hours_until_expiry * 3600).to_i

  # Open the file in binary mode and find its name
  file = File.new(ARGV[0], 'rb')
  path, file_name = File.split(file.path)

  # Upload the file to an S3 object named after the file
  puts "Uploading file: #{file_name}, size: #{file.stat.size} bytes"
  s3 = S3.new
  s3.create_object(BUCKET_NAME, file_name, :data => file)

  # Generate a signed URI to share the S3 object
  puts "URI will be valid for #{hours_until_expiry} hours:"
  puts s3.sign_uri('GET', expiry_time, BUCKET_NAME, file_name)
end

Here is the command you would use to upload a PDF (portable document format) document to S3, and to generate a link that will remain valid for 24 hours:

$ ruby sharefile.rb Documentation.pdf
Uploading file: Documentation.pdf, size: 1527187 bytes
URI will be valid for 24 hours:
http://my-bucket.s3.amazonaws.com/Documentation.pdf
  ?Signature=ixM9AcLac6xy9h5isWoTSk4O7Ws%3D
  &Expires=1194922654
  &AWSAccessKeyId=ABCDEFGHIJ1234567890

When the command has finished, copy and paste the resulting URI into an email message, and the message’s recipients will be able to use the link to download the file for the next 24 hours.

To create a

link that expires in a longer or shorter time, include the number of hours until the link should expire as a second parameter.

# Link will expire in 3 days (72 hours)
$ ruby sharefile.rb Documentation.pdf 72

# Link will expire in 30 minutes (0.5 hours)
$ruby sharefile.rb Documentation.pdf 0.5

Remember to delete the file in S3 when you have finished sharing it.