Learn AWS Serverless Computing

Data storage using S3

Loading the files into S3 is a task that can be achieved in multiple ways—you can get creative depending on the requirements, data volume, file update frequency, and other variables. A few possible ways are as follows:

• A custom shell script (or a set of scripts) that runs AWS CLI commands to send the files to an S3 bucket
• The AWS SDK for your favorite language
• AWS DataSync
• AWS Storage Gateway

We're going to use the AWS CLI directly from the command line to sync a local folder with a newly created bucket:

1. Let's create a bucket that will receive our raw data. As you already know, S3 bucket names must be unique, so update the following command accordingly and choose your unique bucket name. This will create the bucket ...