Python Web Scraping Cookbook

How to do it

We won't parse the data in the planets.html file, but simply retrieve it from the local web server using requests:

1. The following code, (found in 03/S3.py), reads the planets web page and stores it in S3:

import requestsimport boto3data = requests.get("http://localhost:8080/planets.html").text# create S3 client, use environment variables for keyss3 = boto3.client('s3')# the bucketbucket_name = "planets-content"# create bucket, sets3.create_bucket(Bucket=bucket_name, ACL='public-read')s3.put_object(Bucket=bucket_name, Key='planet.html',              Body=data, ACL="public-read")

2. This app will give you output similar to the following, which is S3 info telling you various facts about the new item.

{'ETag': '"3ada9dcd8933470221936534abbf7f3e"', ...