Programming Amazon Web Services

Programming Amazon Web Services

by James Murty
Released March 2008
Publisher(s): O'Reilly Media, Inc.
ISBN: 9780596551797

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Building on the success of its storefront and fulfillment services, Amazon now allows businesses to "rent" computing power, data storage and bandwidth on its vast network platform. This book demonstrates how developers working with small- to mid-sized companies can take advantage of Amazon Web Services (AWS) such as the Simple Storage Service (S3), Elastic Compute Cloud (EC2), Simple Queue Service (SQS), Flexible Payments Service (FPS), and SimpleDB to build web-scale business applications.

With AWS, Amazon offers a new paradigm for IT infrastructure: use what you need, as you need it, and pay as you go. Programming Amazon Web Services explains how you can access Amazon's open APIs to store and run applications, rather than spend precious time and resources building your own. With this book, you'll learn all the technical details you need to:

  • Store and retrieve any amount of data using application servers, unlimited data storage, and bandwidth with the Amazon S3 service
  • Buy computing time using Amazon EC2's interface to requisition machines, load them with an application environment, manage access permissions, and run your image using as many or few systems as needed
  • Use Amazon's web-scale messaging infrastructure to store messages as they travel between computers with Amazon SQS
  • Leverage the Amazon FPS service to structure payment instructions and allow the movement of money between any two entities, humans or computers
  • Create and store multiple data sets, query your data easily, and return the results using Amazon SimpleDB.
  • Scale up or down at a moment's notice, using these services to employ as much time and space as you need
Whether you're starting a new online business, need to ramp up existing services, or require an offsite backup for your home, Programming Amazon Web Services gives you the background and the practical knowledge you need to start using AWS. Other books explain how to build web services. This book teaches businesses how to take make use of existing services from an established technology leader.

Table of contents

  1. Programming Amazon Web Services
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. A Note Regarding Supplemental Files
    3. Preface
      1. What’s in This Book?
      2. Ruby and Interactive Examples
      3. Conventions Used in This Book
      4. Using Code Examples
      5. Safari® Enabled
      6. How to Contact Us
      7. Acknowledgments
    4. 1. Infrastructure in the Cloud
      1. Amazon Web Services for Infrastructure
        1. Signing Up for Amazon Web Services
      2. Thinking Like Amazon
      3. Reality Check
        1. Web Services Are Dependent on a Reliable Network
        2. Amazon Does Not Offer Service Level Agreements for All Its Web Services
        3. Data Security and Privacy
        4. Specialized Infrastructure Requirements
        5. Communication Channels for Fault Reporting and Resolution
      4. Interfaces: REST and Query Versus SOAP
    5. 2. Interacting with Amazon Web Services
      1. REST-Based APIs
        1. HTTP Requests and Responses
        2. REST and Query APIs
        3. XML Documents
          1. Interpreting XML documents with XPath
          2. Hierarchical object structures
          3. Building XML documents
        4. HandlingService Errors
      2. User Authentication
        1. Request Descriptions
        2. Signing Request Descriptions
        3. Common Service Properties
      3. Performing AWS Requests
        1. REST API Implementation
          1. Sending REST requests
          2. Request authentication
        2. Query API Implementation
          1. Request parameters
          2. Sending query requests
          3. Request description
        3. Utility Methods
          1. Time adjustments
          2. Base-64 encoding
          3. Debug logging
    6. 3. S3: Simple Storage Service
      1. S3 Overview
        1. S3 Architecture
        2. Pricing
      2. Interacting with S3
        1. REST API
        2. S3 Implementation Stub
        3. Constructing S3 URIs
      3. Buckets
        1. Bucket Locations
        2. Bucket Names
        3. Create a Bucket
        4. Look Up a Bucket’s Location
        5. List Your Buckets
        6. Delete a Bucket
      4. Objects
        1. Object Keys and Hierarchical Naming
        2. Object Metadata
        3. Objects Are Immutable
        4. Create or Replace an Object
        5. Retrieving Objects
        6. Retrieving Objects Conditionally
        7. Listing Objects
          1. Object details
          2. Truncated listings
          3. Searching
          4. Listing parameters
          5. Listing implementation
          6. Demonstrating object searching
        8. Deleting Objects
        9. Create Objects from a Web Browser Using POST
          1. HTML form for S3 POST
          2. Policy Document for S3 POST
          3. Generate a policy document
          4. Generate a form
      5. Alternative Hostnames
      6. Access Control Lists
        1. ACL Grantees
        2. ACL Permissions
        3. Retrieve a Resource’s ACL
        4. Update a Resource’s ACL
        5. Canned Access Policies
        6. Update a Resource’s Canned ACL
      7. Server Access Logging (Beta)
        1. Configuring Server Access Logging
        2. Logging Files
        3. Retrieve Logging Settings
        4. Modify Logging Settings
        5. Access Permissions for Logging
      8. Signed URIs
      9. Distributing Objects with BitTorrent
    7. 4. S3 Applications
      1. Share Large Files
      2. Online Backup with AWS::S3
        1. AWS::S3 Ruby Library
        2. S3Backup Class
        3. List Backed-Up Objects
        4. Find Files to Back Up
        5. Back Up Files
        6. Delete Obsolete Objects
        7. Putting It All Together
        8. Content-Length Workaround
      3. S3 Filesystem with ElasticDrive
        1. Challenges
        2. ElasticDrive: S3 As a Virtual Block Device
          1. Setup and configuration
      4. Mediated Access to S3 with JetS3t
        1. JetS3t Gatekeeper
          1. Deploy the Gatekeeper servlet
          2. Configure and test Cockpit Lite
          3. Authorization with HTTP Basic
          4. Customizable authorization modules
          5. Implement a custom authorization module
          6. Next steps
    8. 5. EC2: Elastic Compute Cloud (Beta)
      1. EC2 Overview
        1. EC2 Instances
        2. AMIs
        3. The EC2 Environment
        4. Pricing
      2. Interacting with EC2
        1. Query API
        2. EC2 Implementation Stub
      3. Keypairs
        1. List Keypairs
        2. Create a Keypair
        3. Delete a Keypair
      4. Network Security by IP
        1. Allow IP Traffic
      5. Finding Amazon Machine Images
        1. List Available AMIs
      6. Controlling Instances
        1. Stop Running Instances
        2. Starting Instances
        3. List Running Instances
      7. Log In to an Instance
      8. Security Groups
        1. List Security Groups
        2. Creating Security Groups
        3. Deleting Security Groups
        4. Allow Traffic from EC2 Groups
          1. Allow group traffic
        5. Disallow Traffic by IP or Group
      9. Managing and Sharing AMIs
        1. Register an AMI
        2. Deregistering an AMI
        3. Modify Image Attributes
        4. Describe Image Attributes
        5. Reset AMI Attributes
        6. Confirm an Instance’s Product Code
      10. Console Output and Instance Reboot
        1. Console Output Snapshot
        2. Reboot Running Instances
    9. 6. Using EC2 Instances and Images
      1. EC2 Instances in Detail
        1. Linux Kernel
        2. Network Addressing
          1. Dynamic addressing issues
        3. Instance Data
          1. Instance data versioning
          2. Instance Metadata
          3. Instance user data
        4. Performance
      2. Data Management in EC2
        1. Storage Locations
        2. Ephemeral Storage
      3. Modifying an AMI
        1. AMI Security Auditing
        2. Install Amazon’s Bundling Tools
        3. Configure System Services
        4. Bundling an Instance into an AMI
        5. Uploading AMI Files to S3
      4. Registering an AMI
      5. Create an AMI from Scratch
        1. Prepare the AMI Filesystem
        2. Install Fedora 7 Base
        3. Configure Networking
        4. Startup Scripts
        5. Install Additional Software
        6. Cleanup
        7. Bundle and Register Your AMI
    10. 7. EC2 Applications
      1. Dynamic DNS
        1. DynDNS: Free Dynamic DNS Service
        2. EveryDNS.net
      2. On-Demand VPN Server with OpenVPN
        1. OpenVPN Security Group
        2. Start an AMI
        3. Install and Configure OpenVPN on an EC2 Instance
        4. Install and Configure OpenVPN on a Client Computer
        5. NAT Routing
        6. Run OpenVPN as a Service
      3. Web Photo Album with Gallery 2
        1. Launch an EC2 Instance
        2. Install an Apache Web Application Stack
        3. Install the Gallery 2 Application
        4. Data Backup to S3
        5. Restoring Data from S3
        6. Next Steps
    11. 8. SQS: Simple Queue Service
      1. SQS Overview
        1. SQS Architecture
        2. Guidelines for SQS Applications
        3. The Message Life Cycle
        4. SQS Client Roles
        5. Pricing
        6. API Version 2008-01-01
          1. Pricing for API version 2008-01-01
          2. Changes in API version 2008-01-01
      2. Interacting with SQS
        1. Query API
        2. SQS Implementation Stub
      3. Queues
        1. Create a Queue
        2. List Queues
        3. Deleting Queues
      4. Messages
        1. Sending Messages
        2. Receiving Messages
        3. Deleting Messages
        4. Peeking at a Message
        5. Controlling the Visibility of Messages
      5. Queue Attributes
        1. Retrieve Queue Attributes
        2. Modify Queue Attributes
      6. Queue Access Control
        1. Grantees
        2. Permissions
        3. Listing a Queue’s Access Control Settings
        4. Add an Access Control Rule
        5. Revoke an Access Control Rule
    12. 9. SQS Applications
      1. Messaging Simulator
        1. Code Stub
        2. Simulator Receiver
          1. Test the receiver
        3. Running Simulations
        4. Simulation Scenarios
          1. Getting started
          2. Simple simulation
          3. Checking efficiency
          4. Multiple batches
      2. Distributed Application Services with BOTO
        1. The Boto Services Framework
          1. Architecture
          2. Services
          3. Message queues
          4. Message format
          5. Start service script
        2. Toward a Generic Service
          1. Message format
        3. MultiCommandService
          1. VisibilityThread
        4. Bootstrap Service Startup
        5. Building a Distributed Application
          1. Create a custom service instance
          2. Bundle the service instance to an AMI
          3. Testing the MultiCommandService
            1. ImageMagick examples
            2. FFmpeg example
      3. Automated Management of EC2 Instance Pools with Lifeguard
        1. Estimating Load
        2. Managing the Pool
        3. Installation and Testing
        4. Managing a Pool of MultiCommandService Instances
          1. Status notifications from MultiCommandService
          2. A pool manager compatible with MultiCommandService AMIs
          3. Configuring Lifeguard
          4. Automation achieved
    13. 10. FPS: Flexible Payments Service (Beta)
      1. FPS Overview
        1. Transaction Participants
        2. Transaction Walk-Through
        3. Amazon Payments Accounts
        4. Payment Instructions and Tokens
        5. The Co-branded UI (CBUI) Pipeline
          1. Branding
          2. Handling responses
        6. Payment Methods
        7. Pricing
      2. Interacting with FPS
        1. Production and Sandbox Environments
        2. Query API
        3. FPS Implementation Stub
        4. Service Response Messages
          1. Common fields
          2. Service errors
        5. Idempotent Transactions
      3. Managing Your Tokens
        1. Gatekeeper Language Basics
        2. Install Payment Instructions (Create a Token)
        3. View Installed Payment Instructions
        4. Retrieve a Token
        5. List the Tokens in Your Account
        6. Cancel a Token
        7. View a Token’s Usage Limits
      4. Acquiring Third-Party Tokens
        1. Generating CBUI Request URIs
        2. Interpreting CBUI Result URIs
          1. Verifying a CBUI result URI
          2. Using a web browser to obtain CBUI results
          3. Using a simple web server to interpret CBUI results
        3. Acquire a SingleUse Sender Token
        4. Acquire a Multiuse Sender Token
        5. Acquire a Recurring Sender Token
        6. Allow a Third Party to Modify a Token
      5. Pay Now Widgets
        1. The Anatomy of a Pay Now Widget
        2. Generate a Pay Now HTML Form
        3. Interpreting Pay Now Widget Result URIs
    14. 11. FPS Transactions and Accounts
      1. Performing FPS Transactions
        1. Transaction Fees
        2. Transaction References
        3. Perform a Payment
        4. Retrieve Transaction Details
        5. Handling Transaction Results
        6. Retry a Declined Transaction
        7. Get Results
        8. Discard Results
        9. Perform a Refund
        10. Reserve Credit Card Funds
        11. Settle a Reserved Transaction
      2. Account Management and Information
        1. Retrieve Your Account Balance
        2. Account Activity
    15. 12. FPS Advanced Topics
      1. Gatekeeper Language Guide
        1. How FPS Evaluates Gatekeeper Rules
        2. Sample Gatekeeper Documents
          1. Caller instructions
          2. Recipient instructions
          3. Refund sender instructions
          4. Sender instructions
          5. Sender statements for MultiUse or Recurring tokens
        3. Gatekeeper Language Syntax
          1. Assertions
          2. Assignments
          3. Value expressions
        4. Gatekeeper Variables
          1. Private variables
          2. Global variables
      2. Micropayments with FPS
        1. Drawbacks of Micropayment Instruments
        2. The Prepaid Payment Instrument
          1. Acquire a Prepaid Token
          2. Fund a prepaid instrument
          3. Retrieve balance
          4. Retrieve liability
          5. List your prepaid instruments
        3. Postpaid Payment Instrument
          1. Acquire a postpaid token
          2. Retrieve debt balance
          3. Retrieve outstanding debt balance
          4. Settle debt
          5. WriteOffDebt
          6. List your postpaid instruments
      3. Building a Marketplace Application
        1. Charging Fees for Access to Your Marketplace
        2. Acquire a Recipient Token
      4. Subscribing to FPS Event Notifications
        1. Subscribe to Notifications
        2. Unsubscribe from Notifications
    16. 13. SimpleDB (Beta)
      1. SimpleDB Overview
        1. A Simple Database, Not an RDBMS
        2. Pricing
      2. Interacting with SimpleDB
        1. Query API
        2. SimpleDB Implementation Stub
        3. Service Response Messages
      3. Domains
        1. Create a Domain
        2. List Your Domains
        3. Delete a Domain
      4. Items and Attributes
        1. Attribute Parameters
        2. Retrieve an Item’s Attributes
        3. Create or Update an Item
        4. Delete an Item’s Attributes
      5. Representing Data in SimpleDB
        1. Boolean Encoding
        2. Date Encoding
        3. Integer Encoding
        4. Float Encoding
        5. Automated Encoding and Decoding of Values
      6. Performing Queries
        1. Perform a SimpleDB Query
        2. SimpleDB Query Expression Syntax
          1. Predicates
          2. Predicate operators
          3. Set operations
      7. Stock Price Database: A Mini SimpleDB Application
        1. Obtain Historic Stock Price Data
        2. Load Stocks Data into SimpleDB
        3. Query and Retrieve Stock Data
        4. Caching SimpleDB with Memcached
    17. A. AWS Resources
      1. AWS Online Resources
        1. Amazon DevPay (Limited Beta)
      2. Client Tools
        1. Multiple Services
        2. S3
        3. EC2
      3. API Libraries
        1. Multiple Services
        2. S3
        3. EC2
        4. SQS
      4. Third-Party AWS Solutions
        1. Multiple Services
        2. S3
        3. EC2
        4. FPS
    18. B. AWS API Error Codes
      1. S3: Simple Storage Service
      2. EC2: Elastic Compute Cloud
      3. SQS: Simple Queue Service
      4. FPS: Flexible Payments Service
      5. SimpleDB
    19. Index
    20. About the Author
    21. Colophon
    22. SPECIAL OFFER: Upgrade this ebook with O’Reilly

Product information

  • Title: Programming Amazon Web Services
  • Author(s): James Murty
  • Release date: March 2008
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9780596551797